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Abstract 

Estimating  the  parameters  of  filtered  sinusoids  in  noise  from  a  finite  number  of  obser¬ 
vations  has  wide  Air  Force,  DoD,  and  commercial  interest.  Whether  constructing  models 
for  time  series  analysis  or  estimating  the  velocity  of  moving  targets,  the  ability  to  accurately 
estimate  the  frequency,  amplitude,  and  phase  parameters  of  sinusoids  is  of  paramount  im¬ 
portance.  This  research  develops  theoretical  methods  of  analyzing  filtered  sinusoids  in 
noise  and  demonstrates  their  effectiveness  on  the  problem  of  pulsed  sinusoid  parameter 
estimation  for  Electronic  Warfcire  (EW)  applications.  Specifically,  within  the  context  of 
stochastic  modeling  ,  a  new  linear  model,  parameterized  by  a  set  of  Linear  Prediction  (LP) 
coefficients,  is  derived  for  estimating  the  frequencies  of  filtered  sinusoids.  This  model  is  an 
improvement  over  previous  modeling  techniques  since  the  effects  of  the  filter  and  the  co¬ 
efficients  upon  the  noise  statistics  are  properly  accoimted  for  during  model  development. 
In  addition,  the  LP  coefficients  which  minimize  the  squ£ired  error  between  the  system 
model  and  the  observations,  aire  shown  to  be  maximum  likelihood  coefficient  estimates. 
Two  methods  of  estimating  the  coefficients,  based  on  an  iterative  least  squares  (ILS)  and 
iterative  total  least  squares  (ITLS)  solution  to  an  over  determined  system  of  equations, 
Eire  derived  and  shown  to  be  fixed  point  mappings  of  the  coefficients  within  the  domain  of 
allowable  solutions.  Application  of  this  new  linear  model  and  the  ILS  estimator  to  the  EW 
problem  shows  the  ILS  algorithm  outperforms  the  current  estimation  techniques  by  pro¬ 
viding  optimal  frequency  estimates  for  multiple,  filtered,  pulsed  sim^soids  at  lower  signal 
to  noise  ratios.  In  addition,  a  bound  for  the  estimation  error  of  the  LP  coefficients  and  the 
frequencies,  based  on  one  set  of  observations,  is  derived  and  used  to  gauge  the  quality  of  a 
point  estimate  and  establish  associated  confidence  intervals.  The  results  of  this  research, 
whether  taken  individually  or  collectively,  represent  new  contributions  to  the  theory  of 
signal  processing  and  parameter  estimation  and  support  the  m2iny  applications  requiring 
accurate  parameter  estimation  of  both  complex  and  real  filtered  sinusoids  in  noise. 
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PARAMETER  ESTIMATION  FOR 


REAL  FILTERED  SINUSOIDS 

/.  Introduction 

1.1  Problem 

This  dissertation  investigates  the  problem  of  estimating  the  parameters  of  a  linear 
sum  of  filtered  sinusoids  and  noise.  Accurate  estimation  of  the  frequency,  amplitude,  and 
phase  parameters  of  a  signal  containing  a  sum  of  sinusoids  in  noise  is  a  common  task  in 
many  applications  in  the  field  of  applied  science,  engineering  and  statistics  (39:407).  For 
example,  in  the  field  of  forecasting,  it  is  often  desired  to  decompose  a  time  series  into  its 
main  components  of  trend,  irregular,  cyclical  and  seasonal.  This  decomposition  allows  a 
model  of  the  time  series  to  be  constructed  for  time-series  analysis  and  prediction  (23:1-16). 
Since  a  linear  sum  of  sinusoids  in  zero-mean,  normally  distributed,  white  noise  can  be 
used  to  represent  arbitrary  time-series  data  by  these  components  (76),  accurate  model 
construction  relies  on  acciurate  estimation  of  the  corresponding  sinusoidal  parameters. 

Additionally,  various  types  of  radar  systems  require  2iccurate  frequency  measure¬ 
ments  to  determine  the  relative  velocity  of  a  target  and  to  separate  moving  tairgets  from 
stationciry  objects  (79:243-357).  It  is  well  known  the  electromagnetic  wave  reflected  by 
an  object  moving  in  relation  to  the  radcir  will  be  compressed  in  the  direction  of  motion 
(74:464-466).  This  wave  compression  results  in  an  appeirent  change  in  frequency,  known 
bls  the  Doppler  effect,  between  the  transmitted  signed  and  the  reflected  signal  (77:68-148). 
Since  the  amount  frequency  change  is  directly  related  to  the  relative  velocity  of  the  object, 
the  object’s  velocity  can  be  estimated  by  determining  the  difference  between  the  carrier 
frequency  of  the  transmitted  signal  and  the  frequency  of  the  received  signal  (64:727-728). 
Thus,  accurate  frequency  measurements  are  necessary  for  obtaining  high  resolution  velocity 
measurements  and  discriminating  between  moving  objects  and  clutter  (62). 
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Figure  1.1  Electronic  Warfare  Receiver  Block  Diagram 
1.2  Application 

A  critical  military  application  which  requires  accurate  sinusoidal  pzurameter  estima¬ 
tion  involves  Electronic  Warfare  (EW)  receivers.  The  prim2iry  function  of  an  EW  receiver, 
shown  by  the  block  diagram  in  Figuxel.l,  is  to  detect  a  pulsed  radar  signal,  extract  and 
encode  signal  information,  and  pass  this  data  to  a  classification  system  for  radar  system 
identification  (90:7-35). 

The  antenna  and  antialieising  filter  are  usually  designed  to  intercept  radar  signals, 
modeled  by  pulsed  sinusoids  as  depicted  in  Figure  1.2,  typically  in  the  frequency  range 
2  —  18  GHz  (90:10).  The  analog-to-digital  converter  then  digitizes  the  signal  and  passes 
the  resulting  discrete-time  representation  through  a  set  of  Intermediate  Frequency  (IF) 
filters.  These  IF  filters  limit  the  number  of  time-coincident  sinusoids  processed  by  the 
encoder  and  increase  the  Signal-to-Noise  Ratio  (SNR)  of  the  sinusoids  within  the  filter 
passband  (90:13).  The  function  of  a  typical  encoder  is  to  extract  the  sinusoidal  peirameter 
information  present  in  the  signal  and  provide  point  estimates  of  these  parameters  to  the 
classifier.  The  classifier  then  identifies  the  radau-  system  generating  the  signal  by  comparing 
these  estimates  with  stored  radar  system  signal  descriptions. 
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Pulsed  Radar  Signal 


Figiire  1.2  Typical  Radar  Signal:  Digital  representation  of  pulsed  radar  signal  sampled  at 
/,  =  &QGHz.  Signal  parameters:  Amplitude:  b  =  1,  Frequency:  Fa  =  ZGHz,  Phase: 
<t>  =  7r/3. 

Thus,  accurate  estimates  of  the  sinusoidal  parameters  are  critical  for  both  the  con¬ 
struction  of  the  classification  system  and  for  characterizing  the  waveform  parameters  of 
the  pulsed  sinusoids  prior  to  radar  system  identification. 

1.3  Scope 

The  primciry  focus  of  this  dissertation  is  limited  to  investigating  and  developing  algo¬ 
rithms  which  provide  accmate  point  estimates  of  the  parameters  of  real,  filtered  sinusoids 
in  noise.  Dming  algorithm  development,  the  amplitudes,  frequencies  and  phases  of  the 
sinusoids  are  assumed  to  be  deterministic  and  constant  throughout  the  pulse.  Though 
recent  reseairch  has  been  conducted  to  estimate  the  parameters  of  chirp  signals  (41), (72), 
pulsed  sinusoids  stiU  describe  the  majority  of  radar  signals  present  and  will  provide  a  bzisis 
for  examining  more  exotic  waveforms  in  the  future  (90:10).  Additionally,  the  characteris¬ 
tics  of  the  EW  receiver  will  assumed  to  be  known  and  capable  of  being  modeled  as  a  finite 
impulse  response,  linerir,  time-invariant  filter.  It  is  the  simmoidal  pairameters  which  are 
unknown  for  this  application.  Fiurthermore,  prior  to  filtering,  the  noise  will  be  assumed 
to  be  a  zero-mean,  normally  distributed,  uncorrelated,  wide  sense  stationary  random  pro- 
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cess  with  a  known  variance.  Finally,  the  number  of  sinusoids  present  within  any  block 
of  data  will  be  assumed  known  prior  to  providing  parameter  estimates.  Identification  of 
the  number  of  sinusoids  present  in  a  block  of  data  is  an  area  of  research  in  its  own  right 
and  can  be  accomplished  prior  to  the  employment  of  parameter  estimation  techniques 
(1),  (4),  (11),  (18),  (25),  (41),  (93). 

1.4  Contributions 

The  results  of  this  research,  as  delineated  in  Chapter  III  and  Chapter  IV,  are  high¬ 
lighted  via  the  original  contributions  below: 

•  Development  of  the  true  mathematical  model  for  any  digital  system  required  to 
estimate  the  amplitudes,  phases  and  frequencies  of  filtered  sinusoids  in  noise.  Most, 
if  not  all,  digital  signal  processing  systems  implement  a  series  of  filters  to  condition 
the  noisy  data  prior  to  estimating  the  sinusoidal  parameters.  The  model  developed 
as  pcirt  of  this  research  accounts  for  the  effects  of  the  filter  in  developing  parameter 
estimators. 

•  Derivation  of  Maximum  LikeUhood  (ML)  estimators  for  the  amplitude,  phase  and 
frequency  pcirameters  of  filtered  sinusoids  in  noise.  By  correctly  accounting  for  the 
correlation  in  the  noise  due  to  the  effects  of  the  filter,  an  accurate  model  of  the 
probability  density  function  of  the  filtered  signal  is  derived.  ML  estimation  techniques 
are  then  used  to  provide  parameter  estimators.  Simulations  indicate  that  failure 
to  incorporate  the  effects  of  the  filter  into  the  data  model  will  lead  to  suboptimal 
parameter  estimates. 

•  Construction  of  the  true  general  linear  model,  parameterized  by  a  set  of  Linear 
Prediction  (LP)  coeflBcients,  for  estimating  the  frequencies  of  filtered  sinusoids  in 
noise.  By  properly  incorporating  the  effects  of  both  the  filter  and  the  LP  coeflBcients 
upon  the  noise,  this  new  model  represents  the  true  linear  model  for  estimating  the 
LP  coefficients  and  the  sinusoidal  frequencies  from  the  measurements. 

•  Application  of  fixed  point  theory  to  the  estimation  of  the  coefficients  for  the  true 
LP  general  linear  model  eind  development  of  the  Iterative  Leeist  Squares  (ILS)  and 
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Iterative  Total  Least  Squares  (ITLS)  fixed  point  mapping  functions.  Simulations 
indicate  both  methods  provide  minimum  variance,  unbieised  estimates  of  the  LP 
coefficients,  and  consequently,  the  sinusoidal  frequencies,  over  a  wide  range  of  SNRs 
(98). 

•  Derivation  of  the  exact  relationship  between  the  ML  frequency  objective  function 
and  the  true  LP  general  linear  model  for  filtered  sinusoids.  Specifically,  the  set  of  LP 
coefficients  which  minimize  the  squared  error  defined  by  the  LP  general  linear  model 
are  shown  to  provide  ML  frequency  estimates  and  vice  versa. 

•  Development  of  an  exact  ML  estimator,  termed  the  Iterative  Exact  Gradient  Descent 
(lEGD)  algorithm,  for  LP  coefficient  estimation.  Simulations  indicate  the  LP  coeffi¬ 
cients  provided  by  the  lEGD  algorithm  minimize  the  LP  objective  function  thereby 
providing  ML  estimates  of  the  LP  coefficients  and,  consequently,  the  sinusoidal  fre¬ 
quencies. 

•  Proof  that  the  two  most  commonly  used  methods  of  estimating  the  LP  coefficients, 
termed  Iterative  Generalized  Least  Squares  (IGLS)  and  Iterative  Quadratic  Maxi¬ 
mum  Likelihood  (IQML),  are  equivalent  to  the  ILS  and  ITLS  fixed  point  functional 
mappings,  not  minimization  algorithms  ^ls  widely  accepted.  Although  these  two  es¬ 
timators  do  not  minimize  the  LP  objective  function,  simulations  indicate  they  suffer 
only  a  slight  decrease  in  estimation  accmacy,  p2irticularly  at  high  SNRs. 

•  Derivation  of  a  novel  method  for  bounding  the  estimation  error  of  point  estimates  of 
the  LP  coefficients  and  consequently,  the  frequencies,  based  strictly  on  one  realization 
of  the  meaisurement  vector.  Simulations  indicate  the  estimates  of  the  measurement 
error  can  be  used  to  establish  confidence  intervals  for  point  estimates  of  both  the  LP 
coefficients  and  the  frequencies. 

•  Construction  of  the  data  model  which  accmately  describes  the  passage  of  real  sinu¬ 
soids  through  complex  filters.  The  model  shows  the  complex  output  of  a  band-limited 
filter  can  be  decimated  at  twice  the  rate  as  the  corresponding  real  output  without  a 
loss  in  estimation  accuracy. 
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•  Derivation  of  a  complex  form  of  the  LP  general  linear  model  for  estimating  the 
frequencies  of  complex,  filtered  sinusoids  in  noise.  By  properly  incorporating  the 
effects  of  both  the  filter  and  the  LP  coeflBcients  upon  the  noise,  the  complex,  general 
linear  model  developed  represents  the  true  linear  model  for  estimating  the  complex 
LP  coefficients  (97). 

•  Development  of  a  multirate,  channelized,  EW  receiver  based  on  a  nonmaximally 
decimated,  polyphase  realization  of  the  Short  Time  Fourier  Transform  (STFT).  This 
patented  implementation,  (Serial  Number  08/816, 951),  reduces  the  required  encoder 
processing  speed  requirements  by  a  factor  of  16  while  simultaneously  allowing  pro¬ 
cessing  of  multiple,  filtered  sinusoids  (99). 

•  Construction  of  a  new  and  simplified  ILS  algorithm  for  estimating  the  frequency  of 
a  single  complex  sinusoid  in  additive  white  noise.  The  method  developed  exceeds 
the  accuracy  of  other  single  sinusoid  estimation  techniques  and  can  be  implemented 
using  simple  vector  product  operations  (96). 

•  Establishment  of  an  approximate  complex  model  of  the  data  at  the  output  of  the  EW 
receiver.  The  derivations  show  the  decimated  complex  output,  actually  generated 
from  the  sum  of  real  filtered  sinusoids  in  real  noise,  cam  be  approximately  modeled 
as  being  generated  from  the  sum  of  complex  filtered  sinusoids  in  complex  noise. 

The  results  of  this  research,  whether  taken  individually  or  collectively,  represent  a  major 
contribution  to  the  theory  of  signal  processing  and  parameter  estimation.  In  particular,  this 
research  builds  the  bridge  connecting  sinusoidal  frequency  estimation  with  LP  lineju  sys¬ 
tem  modeling.  Finally,  this  dissertation  derives  the  cormection  between  real  and  complex 
sinusoidal  pcirameter  estimation.  The  estimators  constructed  as  a  result  of  this  connection 
will  significantly  improve  the  operational  envelope  of  the  Air  Force’s  next  generation  EW 
receiver. 

1.5  Overview 

Chapter  II  reviews  the  current  literatmre  concerning  real  sinusoidal  parameter  esti¬ 
mation  techniques.  Several  estimation  algorithms  are  briefly  discussed  to  show  that  ciurrent 
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research  in  the  area  of  parameter  estimation  has  neglected  the  effects  of  a  filter  upon  the 
sinusoids  and  noise. 

Chapter  III  derives  estimators  for  the  amplitude,  phase  and  frequency  parameters  of 
filtered  sinusoids  in  white  noise.  After  describing  the  effects  of  a  filter  on  sinusoids  and 
noise,  a  new  data  model  is  developed  and  ML  estimators  for  the  sinusoidal  parameters  are 
constructed  based  on  this  model.  Frequency  estimation  is  then  recast  sis  the  estimation  of 
the  coeflBcients  parameterizing  the  LP  general  linear  model.  After  developing  algorithms 
for  estimating  these  coefficients,  this  chapter  concludes  by  introducing  a  novel  method  to 
assess  the  accuracy  of  the  coefficient  and,  consequently,  the  frequency  estimates. 

Chapter  IV  applies  these  estimation  algorithms  to  a  specific  EW  receiver  architec¬ 
ture.  This  chapter  begins  by  deriving  the  architecture  characteristics  and  showing  how 
substantial  processing  speed  reductions  can  be  obtained  be  employing  multirate  signal 
processing  techniques.  After  modeling  both  the  real  and  complex  forms  of  the  data,  the 
performance  of  the  estimation  algorithms,  within  the  operational  envelope  of  the  receiver, 
is  then  documented  and  shown  to  provide  improved  parameter  estimates. 

Chapter  V  summarizes  the  original  contributions  of  this  dissertation. 
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II.  Literature  Review:  Sinusoidal  Frequency  Estimation 
2.1  Introduction 

This  chapter  reviews  the  current  methods  of  estimating  the  frequencies  of  the  sum  of 
P  real  sinusoids,  s[m],  in  additive  noise,  t«[m],  from  M  observations  i/[m],  using  the  model 

p 

y[m\  =  ^  6*  cos(27r//tm  +  </>*)  +  tulm]  =  s[m]  +  w{m]  (2.1) 

*=i 

Here,  for  0  <  m  <  M  -  1,  the  amplitudes,  {6*},  phases,  and  frequencies,  {/*},  are 

deterministic  but  unknown.  Since  the  amplitudes  and  phases  can  be  found  by  a  linear 
least  squzires  fit  to  the  data  once  the  frequencies  are  calculated  (45),  (71),  most  research 
in  parameter  estimation  has  focused  on  developing  accurate  frequency  estimators.  This 
chapter  begins  by  reviewing  several  methods  of  estimating  the  frequency  of  a  single  sinusoid 
and  concludes  by  examining  the  current  methods  for  P  sinusoids  based  on  linear  predictive 
modeling. 


2.2  Single  Sinusoid  Frequency  Estimation 

In  many  signal  processing  applications,  the  time  and  frequency  components  of  a 
block  of  data  samples  can  be  limited  so  that  only  one  sinusoid  exists  throughout  the  data 
block  (71).  For  example,  a  bandpass  filter  can  be  used  to  limit  the  frequency  range  under 
observation  such  that  the  filtered  output  can  be  considered  as  being  from  a  single  sinusoid 
(90:21). 

One  method  of  estimating  the  frequency  of  a  single  sinusoid  is  via  an  amplitude 
search  of  a  set  of  Discrete  Fourier  Transform  (DFT)  coefficients  followed  by  a  frequency 
interpolation  based  on  these  coefficients  (71) 


f  infe  +  i)i 

M  ^  \Yik  +  l)\  +  \Y{k)\ 


(2.2) 


Here  \Y{k  + 1)|  and  |y(fc)|  represent  the  two  largest  adjacent  DFT  magnitudes.  Since  this 
technique  produces  only  coarse  estimates  which  are  dependent  on  the  block  length,  further 
optimization  techniques  must  be  employed  to  improve  estimation  accmracy  (85). 
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An  alternative  method  for  estimating  the  frequency  of  a  single  sinusoid  employs  the 
conversion  of  the  real  sinusoid  to  its  complex  form.  That  is,  the  complex  form  of  a  sinusoid 
in  noise  can  be  written  as 


it[m]  =  1  4-  tZ;[Tn]  =  +  f'[m]}  (2.3) 

Here,  ui  is  the  radian  frequency  and  {;[m]  =  u/j[Tn]  +  jvilm]  is  a  complex  random  noise 
sequence  described  in  Appendix  G.  One  approach  for  estimating  wj,  and  hence  /i,  relies 
on  extracting  the  instantaneous  phase  of  tI[Tn].  Since  the  frequency  of  an  analog  signal 
can  be  obtained  by  differentiating  its  instantaneous  phase  (5),  a  simil2ir  relationship  exists 
in  the  digital  domain  when  the  instantaneous  phase  of  the  complex  sinusoids  is  given  by 
0j(m)  =  u)im+4>i.  Provided  0i(m)  >  0i(m— 1),  the  frequency  can  be  determined  via  simple 
phase  differencing  (88).  Assuming  61  is  large  relative  to  the  noise  sample  magnitudes,  as 
shown  in  Appendix  J,  ii[m]  can  be  written  jis  (89) 

ii[m]  =  "*+<»• +«'dml)  ^  ^2.4) 

Thus,  large  SNRs  allow  9{m)  to  be  treated  as  a  linear  function  of  the  frequency,  u>i,  and 
the  phase,  <^i,  corrupted  by  cidditive  noise,  t;/[Tn]. 

This  line2ir  phaise  char2icteristic  has  been  exploited  to  provide  several  simple  fre¬ 
quency  estimators.  One  method,  termed  the  Weighted  Phase  Averaging  (WPA)  algorithm, 
extracts  the  phcise  after  summing  N  samples  of  u[m]  to  achieve  the  new  sequence  i/[m], 
(36),  (42), 

,  N-l  I  iV-1 

y[m]  =  ~  ~  -i-  w[m  —  n]  (2.5) 

n=0  n=0 

If  61  is  large  in  comparison  to  the  noise  sample  magnitudes,  y[7Ti]  is  approximated  as  (42) 

y[Tn]  w  (2.6) 

Here,  the  quantities  1/)  =  lji[N  -  l}/2  and  a  =  sin(a;iiV/2)[sin(a;i/2)]~^  are  real  numbers 
and  the  variable  7[m]  =  a~^  vj[m  —  n]  is  random.  Thus,  for  m  =  N  —  —  1, 
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the  resulting  phase  difference,  2(771]  =  9[m\  —  9\m  -  1],  has  the  vector  form 


z  =  27r/i  1  +  [Na]  ^  Gv  j 


(2.7) 


Here  1  is  a  vector  of  ones  while  G  is  a  sparse  matrix  defined  by 


[Gl. 


1  for  t  =  j 
\  —  1  for  j  =  i  +  N 


^  0  otherwise 


(2.8) 


If  lifm]  is  a  zero  mean,  normally  distributed,  independent,  complex  random  variable  with 
variance  cr^,  then,  as  shown  in  Appendix  J,  z  will  be  a  normally  distributed  random  vector 
with  a  mean  of  27r/il  and  covariance  matrix  Kz  =  ^A^a\/26ij  tr^GG^.  As  such,  an 
unbiased,  TninimiiTn  variance  estimator  for  fi  is  found  via  linear  regression  on  2  as  (40:97) 


1^[GG'^]  ^z 
27rl^  [GG^]"'  1 


(2.9) 


Figure  2.1  shows  the  accuracy  of  this  estimator  varies  as  a  function  of  N,  the  SNR, 
and  the  frequency  of  the  sinusoid  (10).  For  N  >  I,  y[m]  can  be  shown  to  be  the  output  of 
a  low  peiss  filter  of  length  N  (64:447);  the  improvement  in  SNR  for  low  frequency  sinusoids 
is  translated  into  a  lower  SNR  threshold.  However,  once  this  threshold  is  reached,  the 
estimator  with  N  =  \  outperformed  the  other  values  of  N  tested  both  as  a  function 
of  SNR  and  frequency.  The  reduction  in  accuracy  for  />  .5  is  due  to  phase  wrapping 
ambiguities.  Provided  the  SNR  is  high  enough  and  the  frequency  range  of  the  input  sinusoid 
is  limited,  this  estimator  will  provide  accurate  frequency  estimates  and  is  relatively  simple 
to  implement  for  real-time  applications  (17),  (47).  In  addition,  a  simple  modification  of 
this  algorithm  allows  the  rate  of  change  of  the  frequency  to  be  estimated  in  addition  to 
the  carrier  frequency  (12),  (34). 

An  alternative  method,  termed  the  Iterative  Phase  Averaging  (IPA)  algorithm,  uti¬ 
lizes  an  iterative  approach  for  estimating  the  frequencies  from  the  phase  (85).  Let  f{k)  be 
the  k*’’'  estimate  of  the  frequency  and  define  the  variable  5*[77i|  =  u[7n]e“''"*"‘.  Then  g*[7n] 
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-IOIoglO(MSE)  -ICMoglO(MSE) 


Frequency  Measurement  Accuracy 


(a)  Sinusoid  Parameters: [6i  =  l,/i  =  .2,0  =  tt/S] 
MSB  calculated  at  SNR  intervals  of  .5dB. 

Frequency  Measurement  Accuracy 


(b)  Sinusoid  Parameters:  [6i  =  1,0  =  tr/S];  SNR  =  lOdB  and 
MSB  calculated  at  frequency  intervals  of  .001  Hz. 


Figure  2.1  WPA  Frequency  Estimation  Accuracy:  MSB  calculated  from  500  indepen¬ 
dent  realizations  of  M  =  25  samples  of  sinusoid  in  noise.  SNR  =  -lOlogioftr^}  and 
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satisfies  the  first  order  difference  equation 


e  •'''*5]k[m  +  1]  -  e^''‘5it[Tn]  =  0 


(2.10) 


where  7*  =  l(cji  —  w*).  For  an  arbitrary  window  function,  t[Tn],  of  finite  length  M,  then 


Af-l 

^  -  t[m  +  l]e’''‘)  ^*[771]  =  0  (2.11) 

m=0 


Using  the  real  and  imaginary  components  to  find  the  phase  yields  the  iterative  estimator 


f{k  +  1)  =  fik)  + 


j  Em=o  +  1]  -  f[m])uHe 

+  1]  +  t[77l])u[77l]e 


(2.12) 


For  a  complex  sinusoid,  the  optimal  window  is  the  parabola  given  by  <[777]  =  m{M—m)  (85). 
The  main  problem  involved  with  using  this  estimator  is  obtaining  a  good  initial  frequency 
estimate.  Typically,  the  initial  estimate  is  obteiined  from  the  Fast  Fourier  Transform  (FFT) 
bin  with  the  maximum  magnitude  (85).  As  shown  in  Figure  2.2  this  algorithm  provides 
accurate  frequency  estimates,  both  as  a  function  of  SNR  and  frequency,  only  when  a  good 
initial  estimate  is  attained.  In  addition,  provided  the  noise  can  be  modeled  as  a  zero  mean, 
complex,  normally  distributed,  uncorrelated  random  variable,  this  algorithm  will  provide 
unbiased  minimum  variance  frequency  estimates  when  provided  with  a  good  initial  estimate 
(85). 


2.3  Multiple  Sinusoid  Frequency  Estimation 

In  general,  for  P  real  sinusoids  in  noise,  accurate  firequency  estimation  can  be  difficult. 
If  the  sinusoids  are  well  separated  in  frequency,  the  DFT  can  be  used  to  provide  accmrate 
frequency  estimates.  However,  if  the  sinusoids  are  not  approximately  orthogonal  over  the 
observation  interval,  DFT  processing  is  ineffective  and  other  techniques  must  be  employed 
(45).  For  these  cases,  direct  mzLximum  likelihood  techniques  have  been  shown  to  provide 
accurate  frequency  estimates  (82).  These  techniques  usually  require  an  iterative,  nonlinear 
multidimensional  search  in  the  frequency  domain  and  are  too  computationally  intense  to 
be  implemented  in  real-time. 
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lOloglO(MSE)  -IOIoglO(MSE) 


Frequency  Measurement  Accuracy 


(a)  Sinusoid  Parameters; [6x  =  l,/i  =  .2,4>  =  tr/8] 
MSB  calculated  at  SNR  intervals  of  .5dB. 


Frequency  Measurement  Accuracy 


(b)  Sinusoid  Parameters:  [bi  =  1,0  =  tr/8];  SNR  =  lOdB  and 
MSB  calculated  at  frequency  intervals  of  .001  Hz. 


Figure  2.2  IPA  Frequency  Estimation  Accuracy:  MSB  calculated  from  500  indepen¬ 
dent  realizations  of  M  =  25  samples  of  sinusoid  in  noise.  SNR  =  -101ogio{c7^}  and 
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Alternatively,  a  simpler  method  for  determining  the  frequencies  can  be  accomplished 
using  linear  predictive  modeling  techniques  (57),  (95).  Let  s[7n]  be  a  real  signal  consisting  of 
the  sum  of  P  real  sinusoids,  or  modes.  A  2P‘'‘  order  polynomial,  A(z),  which  incorporates 
these  modes  can  be  constructed  as  (73:484-485) 

P  2P 

Ao{z)  =  a[0]  JJ(1  -  ZkZ~^){l  -  zj2~‘)  =  ]^a[p]2~P  (2.13) 

)b=l  p=0 

For  P  real  sinusoids,  the  LP  coefficients,  a[p]  ,  are  real  and  symmetric  with  o[p]  =  a[2P— p]; 
the  P  frequencies  present  in  s[Tn]  are  roots  to  A{z)  (8).  In  the  time  dom£iin,  the  polynomial 
becomes  the  Linear  Prediction  (LP)  equation 

2P 

^a[p]s[m -p]  =  0  (2.14) 

p=0 

When  the  sinusoids  are  corrupted  with  noise,  the  LP  equation  is  satisfied  only  in  a  sta¬ 
tistical  sense.  Consequently,  several  algorithms  based  on  algebraic,  iterative  filtering,  and 
maximum  likelihood  techniques,  have  been  developed  to  estimate  the  LP  coefficients  from 
the  observations,  i/[m]. 

2.3.1  Algebraic  Techniques.  One  technique  of  identifying  the  LP  coefficients  uses 
an  extended  LP  model  of  length  L  >  2P  to  account  for  noise  modes  (37)  and  constrains 
the  LP  coefficients  to  satisfy  the  Forward- Backward  LP  (FBLP)  requirements  (45).  The 
resulting  system  of  equations,  formed  by  the  substitution  of  j/[Tn]  for  s[m]  in  the  LP  equa¬ 
tion,  is  then  solved  using  algebraic  techniques  to  obtain  the  L"*  order  polynomial  A{z). 
The  P  roots  of  A{z),  whose  magnitudes  are  closest  to  the  \init  circle,  for  0  <  /  <  .5,  are 
then  used  to  provide  the  frequency  estimates  (43). 

To  construct  the  system  of  equations,  define  o  as  the  vector  of  I-  4- 1  lineair  prediction 
coefficients,  a  =  [l,ao]^,  where  the  vector  Oo  consists  of  the  trimcated  form  of  a  and 
defined  as  Oo  =  [a[l] . . .  a[i]]^.  In  addition,  define  the  M  —  Lhy  L  +  1  forward  observation 
matrix,  Yp,  and  the  backward  observation  matrix,  Yb,  as 

[Ypl,=y[M  +  l-k-l]-  [YB],^,  =  y[M  -L-l-k  +  l]  (2.15) 
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ioT  k  =  1 ...  M  -  L  and  I  =  1 . . .  L  +  1.  Concatenating  Yf  and  Yb  into  a  single  matrix 
gives  the  FB  observation  matrix  YfB  (45) 

ylB  =  [ylyl\  =  [y,,\ynV  (2.16) 

where  is  the  first  column  of  Yfb-  Finally,  define  the  FB  signal  matrix,  Sfb,  in  a 
manner  similar  to  Yfb-  The  resulting  system  of  equations  to  be  solved  becomes 

=  -Vf,  (2.17) 

One  algebraic  technique  for  solving  this  equation  is  based  on  a  Least  Squares  (LS)  so¬ 
lution.  In  general,  the  optimal  LS  solution  requires  the  rank  2P  signal  model  matrix,  Sfb,  to 
be  known  exactly  (43).  Since  the  components  sfm]  are  not  available,  the  method  proposed 
by  Prony  simply  substitutes  y[m]  for  a[m]  to  obtain  the  LS  estimate  (73:406-408,491-493) 

4.  =  -lyfM-'ytlvti.  (2-18) 

For  a  fixed  number  of  measurements,  this  estimator  will  provide  accurate  estimates  only 
for  large  SNRs  (82).  In  addition,  as  the  number  of  observations,  M,  approaches  infinity, 
this  estimator  can  be  shown  to  be  2isymptotically  biased  (14). 

Figure  2.3  shows  the  estimation  accuracy  of  the  LS  algorithm  for  two  real  sinusoids 
in  zero  mean,  independent,  normally  distributed  noise.  For  these  sinusoids,  above  an 
SNR  of  about  15dB  the  estimation  accuracy  improved  as  L  incre^lsed  from  4  to  14.  This 
indicates  the  extra  coefficients  are  adequately  modeling  noise.  However,  this  improvement 
in  estimation  accuracy  comes  at  the  expense  of  rooting  a  large-order  polynomial. 

An  alternate  algebraic  technique  for  obtaining  the  LP  coefficients  from  Equation  2.17 
is  known  as  the  Principal  Eigenvalue  (PE)  method  (44),  (45),  (91).  This  method  provides 
an  estimate  of  the  LP  coefficients  by  first  performing  aji  eigenvalue  analysis  of  the  matrix 
product  \Yj^Yfb]  and  then  providing  the  estimate  of  do  based  on  this  analysis  (45),  (91). 
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Frequency  Measurement  Accuracy 


Figure  2.3  LS  Frequency  Estimation  Accuracy:  MSE  calculated  at  SNR  intervals  of 
.5dB  from  500  independent  realizations  of  Af  =  30  samples  of  two  sinusoids  in  noise. 
Sinusoid  Parameters:  [6i  =  l,/i  =  =  w/S],  [62  =  l./a  =  -22,  (/>2  =  t/3]; 

SNR  =  -10 logic  2(t^;  MSE  =  53p^i  E^l\{fpii)- fp?;  Frequencies  chosen 

from  two  roots  of  A{z)  with  magnitudes  closest  to  unit  circle  for  0  <  /  <  .5. 


The  resulting  estimate  of  the  LP  coefficients  becomes 

2P 

“o  =  £  K^vivj  (2.19) 

1=1 


and 


Here,  the  vectors  Vi  are  the  orthonormal  eigenvectors  of  the  matrix  product 
A|  is  an  associated  eigenvalue  with  Ai  >  A2 . . .  >  A2P  and  {A;}  w  0  for  /  =  2F  +  1 . . .  i. 
The  effect  of  using  this  truncated  representation  is  to  increase  the  SNR  in  the  data  prior 
to  solving  for  do  (44). 


Figure  2.4  shows  the  estimation  accxuracy  of  the  PE  algorithm  for  two  real  sinusoids 
in  zero  mean,  independent,  normally  distributed  noise.  For  these  sinusoids,  above  an 
SNR  of  about  8dB  the  estimation  accuracy  improved  as  L  increased  from  4  to  12.  This 
indicates  the  extra  coefficients  are  adequately  modeling  noise.  However,  this  improvement 
in  estimation  accuracy  comes  at  the  expense  of  rooting  a  large  order  polynomial.  In 
addition,  the  improvement  in  the  SNR  threshold  over  the  LS  algorithm  is  due  to  the  use 
of  only  the  four  largest  eigenvalues  in  the  estimate  of  do. 
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Frequency  Measurement  Accuracy 


Figure  2.4  PE  Frequency  Estimation  Accuracy:  MSE  caJculated  at  SNR  intervals 
of  .5dB  from  500  independent  realizations  of  M  =  30  S2imples  of  two  sinusoids  in 
noise.  Sinusoid  Par2imeters:  [6i  =  l,/i  =  .2,0i  =  :r/8],  [62  =  l,/2  =  -22,02  =  7r/3]; 
SNR  =  -lOlogio  2<t^;  MSE  -  E<=*i(/p(0-/p)^i  Frequencies  chosen 

from  two  roots  of  A{z)  with  magnitudes  closest  to  unit  circle  for  0  <  /  <  .5. 


A  final  algebraic  technique  of  estimating  the  LP  coefficients  is  based  on  a  Total  Least 
Squares  (TLS)  solution  to  Equation  2.17(67),  (30).  That  is,  with  a[0]  =  1,  the  system  of 
equations  to  be  solved  becomes 

yjpga  =  0  (2.20) 

Since  y[m]  =  s[m]  4-  u;[Tn],  the  signal  matrix,  5/j,  is  now  corrupted  with  noise  and  can  be 
represented  as  Yfi,  =  Sfi,  -f  Efi,  where  Ef(,  can  be  interpreted  as  the  matrix  of  errors  (21) 
The  TLS  solution  for  this  over-determined  set  of  equations  is  beised  on  a  Singular  Valued 
Decomposition  (SVD)  of  IVb,  given  by  Yfb  =  UEV^,  so  that  the  estimate  for  a  becomes 


Ef=Yp+ini,0ni,0 


(2.21) 


where  V;  is  the  column  of  V,  the  matrix  containing  the  right  singular  vectors  of  Fps 
and  V{1,1)  is  the  first  element  of  the  I*'*  column  of  V. 
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Frequency  Measurement  Accuracy 


Figure  2.5  TLS  Frequency  Estimation  Accuracy:  MSB  calculated  at  SNR  intervals 
.5dB  from  500  independent  realizations  of  A/  =  30  samples  of  two  sinusoids  in  noise. 
Sinusoid  Parameters:  [5i  =  l,/i  =  .2, =  tt/S],  [62  =  l)/2  =  -22,02  =  ^/3]; 
SNR  =  -lOlogio  2<7^;  MSB  =  Ei=1(/p(0-/p)^  Frequencies  chosen 

from  two  roots  of  A{2)  with  magnitudes  closest  to  unit  circle  for  0  <  /  <  .5. 

Figure  2.5  shows  the  estimation  accuracy  of  the  TLS  algorithm  for  two  real  sinusoids 
in  zero  mean,  independent,  normally  distributed  noise.  For  these  sinusoids,  above  an  SNR 
of  about  6dB  the  estimation  accuracy  improved  as  L  increeised  from  4  to  12.  Again,  this 
indicates  the  extra  coefficients  are  adequately  modeling  noise.  As  with  the  LS  and  PE 
estimates,  this  improvement  in  estimation  accuracy  comes  at  the  expense  of  rooting  a 
large-order  polynomial.  In  addition,  the  TLS  has  a  lower  SNR  threshold  than  either  the 
LS  or  the  PE  algorithm.  This  is  due  to  the  fact  the  TLS  algorithm  accoimts  for  the  noise 
in  the  observation  matrix  as  well  as  the  noise  in  the  observation  vector  (67). 

2.3.2  Iterative  Filtering.  Another  technique  for  identifying  the  LP  coefficients 
exploits  the  property  that  any  regular  stationary  random  process  can  be  represented  as 
the  output  of  a  linear  system  driven  by  white  noise.  This  allows  estimation  of  the  LP  co¬ 
efficients  to  be  accomplished  using  AR  modeling  techniques  (59),  (87:411-412).  A  popular 
method  for  sinusoids  is  based  on  the  Steiglitz-McBride  method  of  linecir  system  identifi¬ 
cation  via  iterative  filtering  (78),  (81),  (82).  The  basic  technique  is  to  iteratively  filter 


the  observations,  2/[m],  with  the  inverse  filter,  1/A(pz),  derived  from  the  LP  coeflScient 
estimates,  to  obtain  a  new  sequence  Here,  p  is  a  convergence  parameter,  0  <  p  <  1, 

used  to  ensure  stability  of  the  inverse  filter  (14),  (69).  New  estimates  of  the  LP  coefficients 
are  then  obtained  from  analysis  of  this  filtered  data.  Ideally,  since  the  roots  of  A{z)  cor¬ 
respond  to  the  frequencies  of  the  sinusoids  present  in  ylm],  the  frequency  response  of  the 
inverse  filter  1/A{pz)  can  be  interpreted  as  2P  narrow  bandpeiss  filters  centered  near  these 
frequencies(69).  Since  the  sinusoidal  component  of  the  filtered  output,  2/^[m],  consists  of 
sinusoids  at  the  same  firequencies  as  the  input  sinusoids,  the  effect  of  the  iterative  filtering 
is  to  increase  the  SNR  at  each  iteration.  The  two  methods  most  often  used  for  LP  coeffi¬ 
cient  identification  are  the  Iterative  Filtering  Algorithm  (IFA)  and  Pcirametric  Filtering 
(PF)  Algorithm. 

The  IFA  estimator  exploits  the  LP  orthogonality  principle  to  find  the  coefficient 
vector  a  (87:339).  Let  i/[m]  be  estimated  from  2P  previous  samples  as 

2P 

vM  =  S  °\p]yh  -  p]  (2-22) 

p=i 

and  let  e[m]  be  the  error  between  the  estimate  and  the  actual  value  of  y[m].  Assuming 
a[0]  =  1,  the  coefficient  vector,  o,  which  minimizes  the  mean  square  error,  E {e[Tn\e[m]} , 
is  found  such  that  the  error  is  orthogonal  to  the  previous  observations  (87:338-341) 

L 

£'{e[Tn]y[m  —  A:]}  =  0  =  ^  a[l]ryy[k  -  1]  (2.23) 

j=o 

for  A:  =  1 . . .  2P  and  ryy[k  —  /]  =  £'{y[A:]y[/]}.  Since  ryy[l]  is  not  known,  it  too  must  be 
estimated  from  the  data  (87:514-577).  The  IFA  estimator  uses  the  Burg  algorithm  with 
p  =  1  to  estimate  Tyy[l\  and  the  LP  coefficients  while  ensuring  the  resulting  polynomial, 
A{z),  is  stable(39:161-171, 228-229).  In  addition,  an  appropriate  nonnegative  window,  such 
as  the  Hamming  or  Optimal  Tapered  Burg  Window,  is  often  employed  to  reduce  the 
dependency  of  this  estimator  upon  the  phase  of  the  sinusoids,  (13),  (35),  (39:231).  The 
LP  coefficients  are  then  iteratively  estimated  using  the  filtered  data  and  the  algorithm 
terminates  upon  convergence  of  the  LP  estimates  (38),  (39:417-419).  This  estimator  is 


2-12 


Frequency  Measurement  Accuracy 


Figure  2.6  IFA  Frequency  Estimation  Accuracy:  MSE  calculated  at  SNR  inter¬ 
vals  of  .5dB  from  500  independent  realizations  of  M  =  30  samples  of  two  sinu¬ 
soids  in  noise.  Parameters:  [6i  =  l,/i  =  .2,(Ai  =  tt/S],  [62  =  l,h  =  -22,  =  t/3]; 

SNR  =  -lOlogio  2^2.  mSE  =  Ep=i  (/p(0  -  U?- 


relatively  simple  to  implement  and  can  be  shown  to  provide  maximum  likelihood  frequency 
estimates  for  a  large  number  of  measurements  (39:417-419). 

Figure  2.6  shows  the  estimation  accuracy  of  the  IFA  estimator  for  two  real  sinusoids 
in  zero-mean,  independent,  normedly  distributed  noise.  For  these  sinusoids,  above  an 
SNR  of  about  7dB,  use  of  the  Optimal  Tapered  Burg  Window  provides  the  most  accurate 
frequency  estimates  over  the  SNR  range  of  7dB  to  20dB.  Regardless  of  the  window  used 
however,  the  flattening  of  the  MSE  curves  indicates  this  estimator  becomes  biased  at  high 
SNRs  for  this  small  number  measurements. 

As  an  alternative  to  the  IFA  correlation  based  approach,  the  PF  algorithm  institutes 
the  sinusoidal  constraints,  a[p]  =  a[2P  —  p],  and  uses  an  LS  methodology  for  estimating 
the  LP  coeflBicients(14),  (50),  (51),  (52),  (53),  (94).  Here,  the  original  observations,  y[m\, 
are  passed  through  a  causal,  infinite  impulse  response  filter  which  is  parameterized  by  a 
set  of  symmetric  scaling  coefficients,  d[pj,  to  obtain  the  filtered  data  yd[m\  as 


ydH  =  -  ^  d\p]ffa\p]yi\m  -  p]  -I-  y\m] 

P=1 


(2.24) 
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For  P  real  sinusoids,  the  scaling  coefficients,  {d[p]},  are  calculated  as,  (53). 


for  g  =  1 . . .  P.  To  obtain  the  LS  estimate  of  the  LP  coefficients,  the  PF  algorithm  imposes 
the  sinusoidal  constraints  on  a  via  a  2P  4-  1  by  P  matrix  B  so  that  a  =  Ba.  Here,  the 
vector  a  is  given  by  a  =  [1,  a[l] . . .  a[P]]^  is  the  set  of  constrained  LP  coefficients.  Defining 
Yc  as  the  constrained  observation  matrix,  Yc  =  Ff(d)B  =  [l/dFc],  where  is  the  first 
column  of  Yc,  yields  the  PF  estimator 

a,  =  -[F/K]-^y/y,  (2.26) 


The  LP  coefficients  2ire  then  iteratively  estimated  using  the  filtered  data  and  the  algorithm 
terminates  upon  convergence  of  the  LP  estimates.  This  estimator  is  relatively  simple  to 
implement  and  can  be  shown  to  be  asymptotically  unbiased  as  M  approaches  infinity  (50). 

Figure  2.7  shows  the  estimation  accuracy  of  the  PF  algorithm  for  two  real  sinusoids 
in  zero  mean,  independent,  normally  distributed  noise.  For  these  sinusoids,  above  an  SNR 
of  about  4dB,  use  of  the  convergence  parameter  of  p  =  .96  provides  the  most  accurate 
frequency  estimates  over  the  SNR  range  of  4dB  to  15dB.  However,  the  flattening  of  the 
MSE  curves  indicates  this  estimator  becomes  biased  at  high  SNRs  when  the  number  of 
data  points  is  finite. 

2.3.3  Maximum  Likelihood.  The  final  technique  considered  for  estimating  the  LP 
coefficients  requires  minimization  of  the  following  objective  function,  J(o),  with  respect  to 
the  LP  coefficients  in  a  (6),  (46),  (68) 

J(a)  =  a^Yj  {A^AY^Ypa  (2.27) 


Here,  a  =  [a[0] . . .  a[2P]]^  while  A  is  the  M  hy  M  —  2P  matrix  of  LP  coefficients 


0 


for  /  <  fc  <  /  +  2P 
otherwise 


(2.28) 
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Frequency  Measurement  Accuracy 


Figure  2.7  PF  Frequency  Estimation  Accuracy:  MSE  calculated  at  SNR  inter¬ 

vals  of  .5dB  from  500  independent  realizations  o{  M  —  100  samples  of  two  sinu¬ 
soids  in  noise.  Parameters:  [6i  =  l,/i  =  .2, 0i  =  rr/S],  [tj  =  I./2  =  -22,  =  ^/3]; 

SNR  =  -10 log, 0  2<72;  MSE  =  ^  "  fpf- 


The  frequencies  found  from  the  LP  coefficients  which  minimize  J(o)  provide  maximum 
likelihood  frequency  estimates  (6),  (63).  Letting  a  =  [a[0] . . .  o[P]]^  and  imposing  the 
sinusoidal  constraints  via  the  2P  -f  1  by  P  -t- 1  matrix  B  yields  the  new  objective  function 


J{a)  =  Q^5^F/(A^A)-'y>BQ 


(2.29) 


The  most  popular  method  of  minimizing  this  equation  is  termed  the  Iterative  Quadratic 
Maximum  Likelihood  (IQML)  algorithm  (6),  (46),  (68).  This  algorithm  constructs  the 
matrix  A  from  the  current  estimate  of  a  and  minimizes  J(q)  with  respect  to  a  by  im¬ 
posing  the  constraint  that  q^q  =  1  (75).  Termination  of  this  algorithm  is  reached  upon 
convergence  of  a.  Though  this  algorithm  involves  2in  eigenvalue  analysis  of  a  P  4-  1  by 
P  -I- 1  square  matrix  at  each  iteration,  relatively  efficient  methods  of  algorithm  implemen¬ 
tation  have  been  developed  (9),  (29),  (46).  Indeed,  this  algorithm  is  currently  the  most 
popular  method  of  estimating  the  frequencies  of  multiple  complex  sinusoids  in  zero-mean, 
independent,  normally  distributed  noise  (6),  (46),  (68),  (63). 
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Frequency  Measurement  Accuracy 


Figure  2.8  IQML  Frequency  Estimation  Accuracy:  MSE  calculated  at  SNR  intervEils 
of  .odB  from  500  independent  realizations  of  Af  =  30  samples  of  two  sinusoids  in 
noise.  Sinusoid  Parameters:  [6i  =  l,/i  =  ■2,4>\  —  tt/S],  [62  =  I1/2  =  -22,02  =  :r/3]; 
SNR  =  -10 logic  MSE  =  ^  Ep=i  E-=i(/p(i)  -  fpf- 

Figure  2.8  shows  the  estimation  accuracy  of  the  IQML  algorithm  for  two  real  sinu¬ 
soids  in  zero-mean,  independent,  normally  distributed  noise.  For  these  sinusoids,  accurate 
frequency  estimates  were  attained  for  SNRs  above  about  lOdB  using  an  initial  estimate  of 
a  =  [1, 0 . . .  0]^.  An  improvement  in  the  SNR  threshold  should  be  possible  if  a  better  ini¬ 
tial  estimate  of  a  were  used.  In  addition,  the  fact  that  optimal  estimates  were  attained  for 
all  SNRs  above  the  threshold  indicates  this  algorithm  can  be  used  as  a  general  frequency 
estimator  for  large  SNRs. 


2.4  Summary 

This  chapter  reviewed  several  frequency  estimators  for  P  real  sinusoids  in  noise.  For 
P  =  1,  relatively  simple  estimators  can  be  constructed  by  extracting  the  phase  of  the 
complex  representation  of  the  sinusoid.  Simulations  show  these  estimators  will  achieve 
optimal  frequency  estimates  provided  the  noise  is  a  zero-mean,  independent,  normally 
distributed,  complex  random  variable. 
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For  P  >  1,  estimation  of  the  frequencies  was  recast  as  the  estimation  of  a  set  of 
LP  coefficients  and  several  methods  for  estimating  the  coefficients  were  examined.  The 
first  technique  utilized  the  FBLP  constraints  to  construct  a  linear  system  of  equations 
based  on  an  degree  polynomial.  Simulations  show  that  above  a  given  SNR  threshold, 
the  resulting  LS,  PE,  and  TLS  estimators  can  provide  relatively  accurate  estimates  of 
the  frequencies  of  sinusoids  in  zero-mean,  normally  distributed,  uncorrelated  noise.  This 
accmracy  comes  at  the  expense  of  determining  the  frequencies  from  a  large  model  order.  In 
addition,  the  optimal  length  of  the  model  depends  on  both  the  number  of  sinusoids  to  be 
estimated  and  their  frequencies.  These  characteristics  prevent  this  technique  from  being 
used  as  a  general  estimator  for  the  frequencies  of  P  real  sinusoids  in  noise. 

The  second  technique  employed  Iterative  Filtering  methods  to  repeatedly  filter  the 
data  prior  to  estimation  of  the  LP  coefficients.  The  LP  coefficients  for  a  minimum  order 
polynomial  were  then  estimated  using  the  IFA  and  PF  methods.  Simulations  show  that 
over  a  range  of  SNRs,  these  estimators  provide  accin-ate  frequency  estimates  for  sinusoids 
in  zero-mean,  normally  distributed,  uncorrelated  noise.  However,  these  accuuracies  are 
dependent  on  algorithm  parameters  and  previous  research  indicates  these  algorithms  can  be 
sensitive  to  the  phases  of  the  sinusoids  (13),  (35).  These  limitations  prevent  this  technique 
from  being  used  as  a  general  estimator  for  the  frequencies  of  P  real  sinusoids  in  noise. 

The  third  technique  used  to  estimate  the  LP  coefficients  was  bcised  on  maximum 
likelihood  techniques.  By  recasting  maximum  likelihood  frequency  estimation  as  a  min¬ 
imization  of  a  nonlinear  fimction,  J(a),  with  respect  to  the  LP  coefficients,  maximum 
likelihood  frequency  estimates  could  be  obtained  as  roots  of  a  minimum  order  LP  poly¬ 
nomial.  The  LP  coefficients  were  then  estimated  using  the  IQML  algorithm  to  minimize 
J(a).  Simulations  show  this  estimator  provides  accurate  frequency  estimates,  above  a 
given  SNR  threshold,  for  sinusoids  in  zero-mean,  normally  distributed,  uncorrelated  noise. 
This  accuracy  comes  as  the  expense  of  an  iterative  eigenvalue  analysis  ofaP-t-lbyP-(-l 
square  matrix. 

This  review  of  the  current  methods  of  frequency  estimation  show  that  the  true  linear 
model  relating  the  LP  coefficients  to  the  frequencies  of  P  real  sinusoids  in  noise  has  not 
been  adequately  established.  This  lack  of  a  true  model  is  evidenced  by  the  many  methods 
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available  for  estimating  the  LP  coefficients  from  the  data.  If  the  true  model  existed,  the 
accuracy  of  the  estimators  would  be  evaluated  against  the  model  itself. 

In  addition,  no  attempt  ha.s  been  made  incorporate  the  statistical  characteristics  of 
the  noise  into  the  construction  of  the  LP  coefficient  estimators.  Each  of  the  estimators 
examined  in  this  chapter  were  evaluated  under  the  assumption  the  noise  was  uncorrelated. 
However,  in  general,  when  analyzing  analog  signals  using  digital  methods,  an  antialiasing 
filter  is  used  to  limit  the  frequency  range  of  the  analog  signals  under  consideration.  Any 
noise  passing  through  the  filter  will  be  correlated  by  the  action  of  the  filter.  Accimate 
estimation  of  the  LP  coefficients  will  require  knowledge  of  how  the  noise  is  correlated. 

Finally,  there  is  currently  no  method  to  gauge  the  accuracy  of  a  point  estimate  of  the 
LP  coefficients,  and  consequently  the  frequencies,  based  strictly  on  a  single  realization  of 
the  data.  Knowledge  that  a  point  estimate  is  unbiased  is  inadequate  for  making  decisions 
based  on  the  estimation  accuracy.  A  bound  on  the  estimation  error  must  also  be  reported. 

This  dissertation  will  employ  stochastic  modeling  techniques  to  construct  the  true 
linear  model  relating  the  LP  coefficients  to  the  frequencies  of  P  real,  filtered  sinusoids  in 
noise.  Once  this  model  is  constructed,  estimators  for  the  LP  coefficients  will  be  derived 
and  their  estimation  accureicies  evaluated.  A  method  for  bounding  the  coefficient  and 
frequency  estimation  error,  based  on  a  single  realization,  will  then  be  developed  and  used 
to  establish  confidence  intervals  for  the  frequency  estimates.  The  estimators  developed 
from  this  research  will  then  be  applied  to  the  Air  Force’s  next-generation  EW  receiver  and 
evaluated  within  the  receiver’s  operational  envelope. 
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III.  Theory:  Sinusoidal  Parameter  Estimation 

3. 1  Introduction 

This  chapter  derives  estimators  for  the  amplitude,  phase  and  frequency  parameters 
of  filtered  sinusoids  in  white  noise.  After  carefully  deriving  a  mathematical  model  describ¬ 
ing  the  effects  of  a  filter  on  sinusoids  and  noise,  Maximum  Likelihood  (ML)  estimators 
for  the  sinusoidal  parameters  are  developed  based  on  this  model.  Estimation  of  the  sinu¬ 
soidal  frequencies  is  then  recast  as  the  estimation  of  the  coeflBcients  of  a  Linear  Prediction 
(LP)  general  linear  model  and  algorithms  based  on  fixed  point  theory  are  developed  for 
estimating  these  coeflBcients.  In  this  dissertation,  the  relationship  between  this  general 
linear  model  and  ML  frequency  estimation  is  established.  Moreover,  two  populcir  ML-type 
methods  for  estimating  the  LP  coefficients  «ire  analyzed  in  depth  and  are  shown  to  be  fixed 
point  estimators.  This  chapter  concludes  by  deriving  a  method  to  estimate  the  variance 
of  a  point  estimate  of  the  LP  coefficients  and,  subsequently,  the  frequencies,  based  solely 
upon  a  single  realization  of  the  measurements  and  knowledge  of  the  noise  variance. 

3.2  Filtered  Data  Model 

This  section  derives  a  mathematical  model  for  a  real  sinusoid  passing  through  a 
linear,  time-invariant  filter.  With  few  exceptions,  most  digital  signal  processing  systems 
operate  on  signals  which  have  been  already  been  passed  through  various  systems.  In  many 
cases,  each  system  can  be  modeled  as  a  linear  time-invariant  filter.  The  model  derived  in 
this  section  shows  how  to  charcicterize  and  account  for  the  effect  of  a  system  filter  upon 
the  parameter  estimation  process  and  is  an  original  contribution  of  this  research. 

3.2.1  Input  Signal.  Let  the  sampled  signal,  x[m],  be  the  sum  of  a  sinusoidal 
signal,  u[m],  and  zero-mean,  independent,  normally  distributed  noise,  T/[m],  so  that 

x[Tn)  =  v[m\  -I-  r][m]  (3.1) 
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Here,  i/[m]  is  defined  as  the  linear  sum  of  P  discrete-time  real  sinusoids 

p  p 

=  X]  blcos{2nfkTn  -f  ^  v^[m\  (3.2) 

*=i  t=i 

where  the  amplitudes,  {6*},  phases,  {<f>l},  and  frequencies,  {/*},  are  assumed  to  be  deter¬ 
ministic  but  unknown  quantities  with  constraints  >  0,  0  <  <  27r,  and  0  <  /*  <  0.5. 

Assuming  there  are  Mo  samples  over  which  Equation  3.1  holds,  in  vector  form 

X  =  u  +  rj  (3.3) 

where  x  =  [x[Mo  -  1] . . .  i[0]]^,  u  =  -  1] . . .  i^[0]]^  and  rj  -  [t)[Mo  -  1] . . .  77(0]]^  are 

Mo-dimensional  vectors  with  real  components.  The  problem  is  to  estimate  the  amplitudes, 
phases,  and  frequencies  of  the  sinusoids  given  the  observations,  x,  and  knowledge  that 
the  noise  vector,  rj,  is  a  zero-mean,  normally  distributed,  random  vector  with  covariance 
Ktj  =  where  I  denotes  the  identity  matrix.  In  general,  this  simple  model  can  not 
be  applied  directly  to  most  practical  systems  since  the  estimation  process  is  typically 
accomplished  after  filtering.  Instead,  the  observations  are  taken  at  the  output  of  a  known 
linear  system  or  filter  which  induces  a  known  covariance  structure  on  the  noise. 

3.2.2  Output  Signal.  Assume  the  linear  system/filter  model  can  be  represented 
cis  a  finite  impulse  response,  linear,  time-invariant  filter  with  N  real  coeflficients,  {/i[n]}  for 
n  =  0 . . .  -  1.  The  frequency  transfer  function,  as  function  of  radian  frequency, 

cj,  is  given  by  (66:193) 

N-l 

/f(e^“)  =  ^  =  |if(eJ")|e^'*<")  (3.4) 

n=o 

where  u  =  2nf.  With  input  x[m],  for  W  —  1  <  m  <  Mg  —  1,  the  steady  state  output  yfm] 
will  be  real  and  can  be  represented  as  the  sum  of  the  filtered  sinusoid,  s[m],  and  filtered 
noise,  w[m]  (66:192-205).  That  is 

N-l  N-l 

y[m\  =  a[m]  -h  u;[m]  =  ^  /i[n]t/[m  —  n]  -I-  ^  /i[n]77[n  —  m]  (3-5) 

n=:0  n=0 
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Alternatively,  since  j/[m]  is  defined  to  be  in  the  steady  state,  the  sinusoidal  component  of 
2/[m],  denoted  s[Tn],  becomes 

p 

s[Tn]  =  E  6*  cos(27r/jfcm  +  0*)  (3.6) 

k=l 

Here  6*  =  and  0*  =  +  <t>l  define  the  deterministic  effects  of  the  filter 

on  the  input  sinusoidal  amplitudes  and  phaises.  Now  assume  there  are  M  samples  of  the 
steady  state  output  with  M  =  Mo  —  iV  +  1  >  0,  and  define  the  Mo  by  M  Toeplitz  filter 
matrix  H  as  (40:570) 

h[Q]  ...  h[N-l]  0  0  ...  0 

0  h[Q]  . . .  h[N  -  1]  0  ...  0 

:  :  :  :  0  :  0 

0  0  ...  0  h[0]  ...  h[N-l] 

For  m  =  AT  -  1 . . .  Mo  -  1,  the  output,  j/[m],  can  be  written  in  vector  form  as 

y  =  H^x  =  =  s  +  w  (3.8) 

where  y  is  defined  as  an  M-dimensional  vector  with  real  components 

y  =  [y[Mo  -  l]...y[N-l]f 

while  8  and  w  cire  defined  in  a  maimer  similar  to  y.  The  problem  now  is  to  estimate  the 
amplitudes,  {6*},  phases,  {</>*},  and  frequencies,  {/*},  of  the  sinusoids  given  the  observa¬ 
tions,  y.  To  accomplish  this  estimation,  a  model  for  the  observations  will  be  developed 
in  terms  of  the  frequencies  present  in  the  signal  and  a  set  of  scaling  coefficients  which  are 
directly  related  to  the  amplitudes  and  phases. 

3.2.3  Model  Development.  Since  Mg  >  M,  the  columns  of  H  will  be  linearly 
independent,  and  the  noise  vector,  w,  becomes  a  zero-mean,  normally  distributed,  corre¬ 
lated  noise  sequence  with  covariance  matrix  Kyg  given  by  Kyj  =  cr'^H^H  (86:56).  This 
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M  by  M  square  matrix  is  assumed  to  be  nonsingular  so  that  K:^  exists.  The  observation 
vector,  y,  can  now  be  described  as  a  normally  distributed  random  vector  with  a  mean  vec¬ 
tor  my  =  8  and  a  covariance  matrix  Ky  =  Ky].  Using  trigonometric  identities  (100:457), 
Equation  3.5  can  be  rewritten  as 

p 

2/[m]  =  s[m]  -I-  u;[Tn]  =  cos(mcJit)  -  sin(T7ia;jfc)  -f  wlm]  (3.9) 

fc=i 

where  the  scaling  coefficients,  6J  =  bkSm{(i>k)  and  bl  =  6tcos((/)t),  are  directly  related  to 
the  phases  and  amplitudes  (40:198).  That  is 


<t>k 


tan-^  \b{/bl\ 

TT  -  tan-i  16J/6J1 
TT  +  t&n-^  \bi/bi\ 

,  27r  -  tan-i  |6i/6|| 


if  6*  >  0;  >  0 

if  >  0;  5^  <  0 
if  bi<0;bl<0 
if  <  0;  bl  >  0 


(3.10) 


Now  define  A  as  the  deterministic  signal  matrix  as  A  =  [A1IA2]  where  the  M  by  P  matrices, 
Ai  and  A2,  are  constructed  as 


Ai 


cos{(jJi[Mo  -  1]) 

cos{ui[N  -  1]) 


cos{u) p[Mo  -  1]) 

cos(u;/>[jV  -  1]) 


(3.11) 


while 


A2 


-sin(u;i[Afo  -  1|) 


-sin(u;p[Mo  -  1]) 


[  —  sin(wi[iV  —  1])  ...  - sin(wp[iV  -  1])  J 

With  scaling  vectors  b,  =  [61  sin((/>i) . . .  6psin(<^p)]^  and  =  [biCos{(f>i) 
a  vector  form  of  Equation  3.9  can  be  written  as 


(3.12) 

bp  cos{<l)p)f, 


y  =  Ai6c  -I-  A26,  +  w  =  Ab  +  w 


(3.13) 
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where  b  is  the  vector  of  scaling  coefficients,  =  [bj;  6^]  and  the  2P  columns  of  A  are 
linearly  independent.  This  is  equivalent  to  viewing  y  as  a  set  of  observations  generated 
from  the  deterministic  system  Ab  corrupted  by  the  colored  noise  vector  w.  The  problem 
of  estimating  the  frequencies,  phases,  and  amplitudes  has  been  transformed  into  the  prob¬ 
lem  of  estimating  the  signal  matrix  A  and  the  scaling  coefficients  b^  and  b,.  Once  the 
scaling  coefficients  have  been  found,  the  aissociated  phcises  and  amplitudes  can  be  found 
by  Equation  3.10. 

3.2.4  Section  Summary.  This  section  derived  the  mathematical  model  for  any 
system  required  to  estimate  the  amplitudes,  phases  and  frequencies  of  filtered  sinusoids. 
Specifically,  once  the  steady  state  has  been  reached,  the  output  of  a  filter  due  to  an  input 
consisting  of  a  linear  sum  of  P  sinusoids  in  zero  mean,  independent,  normally  distributed 
noise  can  be  represented  as  the  output  from  a  deterministic  system,  Ab,  corrupted  by 
zero  mean,  normally  distributed  noise  with  a  covariance  matrix,  Kw  Estimation  of  the 
sinusoidal  parameters  has  been  recast  as  the  estimation  of  the  deterministic  system  Ab. 
Methods  for  estimating  the  parameters  of  this  model  will  be  derived  in  the  next  section. 

3.3  Sinusoidal  Parameter  Estimation 

This  section  derives  algorithms  for  estimating  the  amplitudes,  frequencies  and  phrises 
of  filtered  sinusoids  in  noise.  After  reviewing  estimation  background  in  general,  this  section 
covers  the  principles  of  maodmimum  likelihood  estimation  and  employs  these  principles  to 
obtaun  parameter  estimators  for  filtered  sinusoids. 

3.3.1  Estimation  Background.  In  general,  estimating  the  pairameters  of  P  sinu¬ 
soids  in  noise  deals  with  inferring  the  values  of  the  unknown  parameters  from  a  set  of  obser¬ 
vations  (87:279).  That  is,  from  Equation  3.9,  M  measurements  are  obtained  which  contain 
the  P  sinusoids  embedded  in  noise.  Since  noise  is  a  random  quantity,  each  observation, 
y[m],  becomes  a  random  vairiable.  Consequently,  y  becomes  a  random  vector.  By  defining 
d  as  a.  vector  form  of  the  parameters,  6^  =  [Sj,  O'^]  =  [bj  . . .  6^;  (^1 . . .  <pp;  f\...  fp]^, 
an  estimator,  or  function  of  the  data  5i(*),  can  to  be  constructed  which  assigns  a  value  to 
6,  as  0  =  gi{y),  for  each  realization  of  y.  Here,  9  is  called  the  point  estimate  of  0  (61).  As 
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a  function  of  a  random  vector,  y,  the  parameter  vector,  6,  becomes  a  random  vector  with 
statistical  characteristics  defined  by  the  Probability  Density  Function  (PDF)  of  y  and  the 
form  of  Qii*)-  The  accmacy  of  subsequent  estimates  are  dependent  primarily  on  the  bicis 
and  variance  of  the  estimates  generated  by  the  estimator. 

Formally,  an  estimator  gi{*)  is  said  to  produce  unbiased  estimates  of  the  true  pa¬ 
rameter  vector,  6,  if  the  expectation  of  9  is  equal  to  9  (27:204).  That  is 

E{9}  =  9  (3.14) 

where  £?{*}  denotes  the  expectation  operator.  Thus,  on  average,  the  unbiased  estimator 
will  provide  the  true  value  of  9  (40:15-23).  In  addition,  gi{*)  is  said  to  be  the  minimum 
variance  estimator  if  the  variance  of  each  component  of  9  is  less  than  the  variance  produced 
by  any  other  estimator  (40:23).  This  implies  gi{y)  is  the  minimum  variance  estimator  if 

Kq  -  >  [0]  for  i  ^  j  (3.15) 

Here  denotes  the  covariance  matrix  of  the  estimate  provided  by  gi{y)  and  the  term 
>  [0]  is  interpreted  as  meaning  the  resultant  matrix  is  positive  semidefinite  (40:44).  If  a 
minimum  variance  unbiased  estimator  can  be  found,  on  average,  the  estimates  obtained 
will  be  closer  to  the  true  value  than  those  provided  by  any  other  estimator. 

A  critical  step  in  the  development  and  analysis  of  point  estimators  is  to  accurately 
model  the  PDF  of  the  data  (40:7).  Since  y  is  a  random  vector  which  contains  the  parame¬ 
ters  to  be  estimated,  the  PDF  of  y  is  said  to  be  peirameterized  by  0  for  0  6  ©  and  written 
as  p(y;  9)  to  show  this  dependency.  Here,  0  is  called  the  peirameter  space  and  contains  the 
restrictions  on  the  allowable  values  of  9  (27:201).  Furthermore,  since  the  actual  value  of  9 
affects  the  probability  of  observing  y,  selection  of  the  proper  PDF  describing  the  data  is 
critical  in  deriving  a  good  estimator.  From  the  data  model  of  Equation  3.13,  the  measure¬ 
ment  vector,  y,  is  an  Af-dimensional,  normally  distributed  random  vector  with  a  known 
covariance  matrix  resulting  firom  the  system  filter  matrix,  H.  The  associated  PDF  of  y  is 
thus  parameterized  by  the  scaling  coefficients,  b,  and  the  frequencies,  f  f  =  [fi  ■  ■  ■  /p]^. 
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That  is 


p{y-e)  =  [2n]-^^\Kw\-^exp[-^{y  -  AbfK^^iy  -  A6)]  (3.16) 

where  0^  =  [6^,/^].  The  accuracy  of  any  estimator  will  be  strongly  dependent  on  how 
well  the  estimator  incorporates  this  knowledge  of  the  PDF  into  its  development. 


3.3.2  Estimation  Accuracy.  To  analyze  the  performance  of  any  estimator,  it  is 
useful  to  have  a  limit  or  bound  which  indicates  the  best  estimation  accuracy  any  unbiased 
estimator  may  obtain  from  the  available  data  (71).  In  particular,  for  sinusoidal  signals 
embedded  in  zero  mean,  normally  distributed  noise,  the  Cramer  Rao  Lower  Bound  (CRLB) 
provides  a  limit  on  the  accuracy  any  unbiased  estimator,  6,  can  attain.  Hence 


Kg  -  >  [0] 


(3.17) 


Here,  ^{6)  is  the  Fisher  Information  matrix  defined  by  (40:44) 


\d\np{y,d) 
dSi 


\dlnp{y]0) 


d9j 


(3.18) 


Since  the  observation  vector,  y,  is  normally  distributed  with  a  mean,  rny,  and  covariance 
matrix,  Kw^  the  Fisher  Information  matrix  can  be  shown  to  be  given  by  (40:47) 


= 


dmy''^ 


d9i  J 


[K 


w\ 


dmi 


d9, 


(3.19) 


For  sinusoids  in  noise,  the  CRLB  will  be  a  function  of  the  frequencies,  phases,  and  am¬ 
plitudes  present  in  the  signal.  Figure  3.1  shows  the  CRLB  for  frequency  estimates,  as  a 
function  of  the  signal  frequency,  for  one  and  two  sinusoids.  In  Figure  3.1a,  for  the  non- 
filtered  signal,  the  frequency  estimation  accuracy  will  be  relatively  constant  except  for 
extremely  small  (/  <  0.001)  or  extremely  high  (/  >  0.499)  frequencies.  For  the  filtered 
sinusoid,  the  estimation  accuracy  will  be  optimal  within  the  filter  pcissband.  Figure  3.1b 
shows  a  contour  plot  of  the  combined  CRLB  for  the  frequencies  of  two  sinusoids  in  zero- 
mean,  normally  distributed,  independent  noise.  For  this  small  number  of  measurements. 
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M  =  32,  this  figure  indicates  the  estimation  accuracy  of  any  estimator  will  decre2ise  dra¬ 
matically  for  closely  spaced  sinusoids,  (|/i  —  /2I  <  -02). 

3.3.3  Estimator  Evaluation.  Since  the  CRLB  is  a  complicated  function  of  the 
frequencies,  phases  and  amplitudes  of  the  sinusoids,  there  is  no  closed  form  expression 
for  determining  whether  an  estimator  which  will  achieve  the  bound  for  any  8irbitrary  G 
(39:414)  .  Consequently,  the  performance  of  any  estimator  must  be  evaluated  numerically 
via  Monte  Ceirlo  experiments  (40:164-167).  For  this  dissertation,  the  performance  of  an 
estimator  will  be  determined  by  conducting  500  independent  trials  at  each  free  variable  of 
interest  and  determining  the  combined  Mean  Square  Error  (MSE)  for  each  parameter  to 
be  estimated. 

-  P  500 

MSE  =  ^  E  (3.20) 

p=i «=i 

The  resulting  experimentally  obtained  MSE  will  then  be  compeired  to  the  associated  CRXB. 
Though  various  techniques  exist  to  construct  estimators  based  on  the  PDF  given  by  Equa¬ 
tion  3.16,  this  dissertation  will  employ  the  maximum  likelihood  estimation  technique.  The 
estimators  provided  by  this  technique  are  relatively  simple  to  develop  for  signals  embedded 
in  normally  distributed  noise  and  produce  the  approximate  minimum  variance  unbiased 
estimates  for  a  moderate  number  (Af  >  30)  of  observations  (40:157-198). 

3.3.4  Maximum  Likelihood  (ML)  Estimation.  The  Maximinn  Likelihood  (ML) 
estimation  technique  is  based  on  the  aissumption  that  a  peirticular  pau'ameter  vector,  6i, 
generates  different  sets  of  observations  than  any  other  parameter  vector,  Gj.  Thus,  any 
given  observation  vector,  y,  is  more  hkely  to  have  been  generated  by  one  particulzu:  pciram- 
eter  vector  Gi,  than  any  others  (60:147).  Obtaining  an  ML  estimator  involves  specifying 
the  likelihood  function  and  finding  the  values  of  the  parameter  vector  w'hich  maximize  this 
function  for  a  fixed  observation  vector  (60:49).  In  general,  when  viewed  cis  a  function  of 
G  for  a  fixed  observation  vector,  the  PDF  defined  in  Equation  3.16  is  termed  a  likelihood 
function  (40:29).  Thus,  for  a  fixed  observation  vector  y,  the  ML  estimate  for  G  maximizes 
p(y;  G).  Equivalently,  since  the  natural  logarithm  is  a  monotonically  increasing  function, 
the  ML  estimate  for  G  maximizes  ln{p(y;0)};  the  log  likelihood  function. 
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Frequency-f2  (Hz) 


CRLB  For  Frequency 


(a)  Sinusoid  Parameters:  [6°  =  1,<^J  =  ir/3],  Filter  Parameters: 
Center  Frequency:  fc  =  21;  Bandwidth:  /b  =  .2;  Length:  N  =  32. 


0.05  0.1  0.15  0.2  0.25  0.3  0.35  0.4  0.45 

Frequency-fl  (Hz) 


(b)  Sinusoid  Parameters:  [hf  =  l,<^i  =  7r/3],  [62  =  —  4x/3]. 

Figure  3.1  CRLB  for  Sinusoids  in  Noise;  Bound  obtained  for  one  sinusoid  in  (a) 
and  two  sinusoids  in  (b).  Variance  bounded  as  function  of  frequency  at  fixed 
SNR  of  lOdB  and  block  length  M  =  32.  For  two  sinusoids,  contours  plotted  as 
Vor  =  .5[Var{/i}  +  Var{/2}].  SNR  calculated  as  SNR  =  -101ogio(2o’^). 
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Furthermore,  if  a  =  g{d),  where  g{*)  is  an  arbitrary  fc-dimensional  invertible  function 
of  9,  the  ML  estimate  for  a  is  given  by 


^ML  —  gi^Ml)  (3-21) 

This  property,  known  as  the  Invariance  Property  of  ML  estimates,  states  that  any  invertible 
function  of  an  ML  estimate  will  also  produce  an  ML  estimate  (40:182). 

3.3.5  Parameter  Estimators.  From  the  PDF  of  y  given  by  Equation  3.16,  with 
Kw  assumed  known,  as  shown  in  Appendix  B,  the  ML  estimate  for  6  can  be  obtained  by 
minimizing  the  following  objective  function,  J{6),  with  respect  to  the  parameters  in  9: 

J{9)  =  [y-  A61^  [y  -  A6]  (3.22) 

Minimizing  this  function  with  respect  to  9  can  be  accomplished  by  minimizing  with  respect 
to  the  scaling  coefficients,  b,  and  frequencies,  /,  individually.  For  a  fixed  ML  estimate  of 
/,  the  ML  estimator  for  b  is  unique  and  found  as  (22),  (40:186) 


b{ML)  =  [A^/Cti/A]  ‘  A^K^^y 


(3.23) 


In  addition,  the  statistical  characteristics  of  b  become  (60) 

E{b{ML)}=b  and  (3.24) 

This  estimator  is  known  as  the  Best  Line2ir  Unbiased  Estimator  (BLUE)  for  a  fixed  set  of 
frequencies  (40:140).  The  ML  estimate  for  /,  on  the  other  hand,  minimizes  the  following 
objective  fimction  (40:186): 

j{f)  =  y^K:^y  -  y^K^^A  [A^ATji/A]"^  A^K:^y  (3.25) 

This  objective  function,  J(/),  termed  the  ML  frequency  objective  function,  is  a  strongly 
nonlinear  function  with  respect  to  /.  Since  the  ML  estimates  of  the  scaling  coefficients, 
6,  Eire  defined  as  a  function  of  the  frequencies,  /,  ML  estimation  of  the  frequencies  is 
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critical  for  obtaining  accurate  parameter  estimates.  The  global  minimization  of  this  non¬ 
linear  function  typically  involves  a  computationally  intensive  search  in  the  P-dimensional 
frequency  domain  and  various  techniques  have  been  tried  to  accomplish  this  search  (80). 

3.3.6  ML  Frequency  Estimation.  Since  the  term  y^K^y  is  a  positive  number, 
minimizing  J{f)  can  be  accomplished  by  maximizing  the  following  objective  function 

Mf)  =  y'^Kij}A[A^KijA]-^A'^K^^y  (3.26) 

One  convenient  method  of  ciccomplishing  this  maximization  involves  the  general  method 
of  iterative  gradient  ascent 


(3.27) 


Here,  each  element  of  the  gradient  is  found  as 


[VJi(/)]  = 


dMf) 


df 


dMf) 

dh 


and  £)  is  the  Hessian  of  Ji(/). 


{D]i3  = 


d^Mf) 


dfdf 


^Mf) 

dfidfi 


(3.28) 


(3.29) 


In  the  area  of  signed  processing  and  numerical  analysis,  this  technique  is  known  as  the 
Newton- Raphson  Technique  (40:187)  while  in  the  area  of  pattern  recognition,  it  is  known 
as  the  Conjugate  Greulient  Method  (3:274-275).  Regeurdless  of  the  name,  as  shown  Ap¬ 
pendix  E,  the  expression  for  the  gradient  becomes  relatively  simple  while  the  calculation 
of  the  Hessian  is  computationally  intensive.  An  alternative  method  of  maximizing  Ji  (/) 
can  be  foimd  by  using  the  direct  gradient  ascent  maximization  technique  (7) 


S{i  +  1)  =  S{i)  +  r?(i)VJ(/)  7(0[/(i)  -  /(i  -  1)]  (3.30) 
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Here,  the  rate  of  convergence  is  controlled  by  the  adaptive  gain  term  rj{i)  and  the  adap¬ 
tive  momentum  term  7(1).  As  with  most  iterative  optimization  techniques,  there  is  no 
guarantee  the  algorithm  will  ever  converge.  Even  if  it  does  converge,  there  is  no  guarantee 
the  global  maximum  has  been  attained,  especially  for  nonconvex  functions.  The  solution 
achieved  will  only  provide  a  local  maximum  and  the  accuracy  of  the  estimate  will  depend 
greatly  on  the  location  of  the  initial  guess. 

Figure  3.2  shows  the  convergence  of  this  iterative  technique  for  one  and  two  filtered 
sinusoids.  Since  the  objective  function,  Ji{f),  contains  more  than  one  local  maximum, 
the  initial  estimate  was  obtained  by  evaluating  Jiif)  at  equally  spawred  test  points  and 
choosing  the  frequency  with  the  largest  Ji{f).  In  both  instances,  the  method  of  direct 
gradient  ascent  found  the  global  maximum  in  less  than  30  iterations. 

Figure  3.3,  Figure  3.4,  and  Figure  3.5  show  the  estimation  accuracy  attained  using 
ML  estimates  of  the  parameters  of  one  and  two  filtered  sinusoids.  This  figure  shows  the  ML 
parameter  estimates  will  achieve  the  CRLB  ,  above  threshold,  only  when  the  correlation  in 
the  noise  is  correctly  modeled.  Assumptions  that  the  noise  is  uncorrelated  (white)  leads  to 
an  incorrect  model  and  suboptimal  estimates.  Since  most  digital  signal  processing  systems 
incorporate  an  antialiasing  filter  as  a  minimum,  the  noise  will  always  be  correlated.  The 
results  of  these  figures  show  the  fallacy  of  using  a  white  noise  model  to  construct  the  CRLB 
for  filtered  signals. 

3.3.7  Section  Summary.  This  section  derived  ML  estimators  for  the  amplitude, 
phase  and  firequency  parameters  of  P  filtered  sinusoids  in  noise  and  represents  an  original 
contribution  of  this  reseaurch.  Specifically,  by  correctly  accounting  for  the  correlation  in 
the  noise  due  to  the  effects  of  the  filter,  an  accurate  PDF  model  of  the  measurements 
was  derived.  Manipulation  of  the  objective  function  derived  from  this  PDF  showed  ML 
frequency  estimates  provide  ML  estimates  of  the  sinusoidal  amplitudes  and  phaises.  Sim¬ 
ulations  indicate  the  method  of  gradient  ascent  can  be  used  to  provide  the  ML  frequency 
estimates  provided  the  PDF  model  accxirately  describes  the  correlation  in  the  noise  due 
to  the  filter.  Failure  to  incorporate  the  filter  effects  in  the  ML  model  will  lead  to  subopti- 
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Maximizing  Frequency  Objective  Furxrtlon 


(a)  Sinusoid  Parameters:  [bj  =  l,/i  =  .191,  =  3t/8],  SNR  =  OdB. 

Maximizing  Frequency  Objective  Function 


(b)  Sinusoid  Parameters:  [bf  =  l,/i  =  .207,^1  =  Stt/S],  [bj  =  l,/2  =  .227,  (Af  =  4tr/3] 

and  SNR  =  .5dB. 

Figure  3.2  Maximizing  ML  Frequency  Objective  Function:  J{f)  calculated  as 
function  of  frequency  at  fixed  SNR  and  block  length  M  =  32.  Initial  estimates  made 
by  choosing  /  with  Izu’gest  J{f)  from  uniform  samples  across  filter  bandwidth.  Fil¬ 
ter  Parameters:  Center  Frequency:  fc  =  .21;  Bandwidth:  fs  =  .2;  Length:  N  =  32; 
SNR  calculated  as  SNR  =  — 101ogio(2<T^). 
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-10(og10(MSE)  -10(og10(MSE) 


Frequency  Measurement  Accuracy 


Frequency  Measurement  Accuracy 


Figure  3.3  ML  Frequency  Estimation  Accuracy:  MSE  calculated  at  SNR  intervals  of 
.5dB  from  500  independent  realizations  of  M  =  32  samples  of  sinusoids  in  noise.  The 
incorrect  model  assumed  Kw  =  while  the  correct  model  used  Kw  =  Fil¬ 

ter  Parameters;  Center  Frequency:  fc  =  .21;  Bandwidth:  /b  =  .2:  Length:  N  =  32; 
SNR  calculated  as  SNR  =  -101ogio(2<T2);  MSE  =  Ep=i  ES(/p(»)  -  /p)'- 
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50 


Amplitude  Measurement  Accuracy 


AmpMItude  Measurement  Accuracy 


Figure  3.4  ML  Amplitude  Estimation  Accuracy:  MSE  calculated  at  SNR  in¬ 

tervals  of  .5dB  from  500  independent  realizations  of  M  =  32  samples  of  si¬ 
nusoids  in  noise.  The  incorrect  model  assumed  Kw  =  while  the  correct 
model  used  Kw  =  Filter  Parameters:  Center  Frequency:/^  =  .21;  Band¬ 
width:  =  Length:iV  =  32;  SNR  calculated  as  5Ni?  = -10  logio(2<7^)  and 

MSE=jkoEUZZ\iUii)-fp)^- 
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Phase  Measurement  Accuracy 


Phase  Measurement  Accuracy 


Figure  3.5  ML  Phase  Estimation  Accuracy:  MSE  calculated  at  SNR  intervals  of  .5dB 
from  500  independent  realizations  of  Af  =  32  samples  of  sinusoids  in  noise.  The  incor¬ 
rect  model  assumed  Kw  =  while  the  correct  model  used  Kw  =  Filter 

Parameters;  Center  Frequency;  /e  =  .21;  Bandwidth;  fs  =  .2;  Lenrth;  N  =  32;  SNR 
calculated  as  SNR  =  -101ogio(2(7=')  and  MSE  =  Etxifpii)  “  fp?- 
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mal  parameter  estimators.  The  next  section  introduces  an  alternate  method  for  frequency 
estimation  based  on  linear  prediction  modeling. 

3.4  Linear  Prediction  (LP)  Modeling 

This  section  develops  an  alternate  method  of  estimating  the  frequencies  of  pulsed 
sinusoids  based  on  Linear  Prediction  (LP)  modeling  techniques.  Specifically,  this  section 
derives  the  true  linear  model,  parameterized  by  the  LP  coefficients,  for  estimating  the 
frequencies  of  P  sinusoids  in  noise  and  is  an  original  contribution  of  this  resezurch.  Using 
this  model  and  properly  accounting  for  the  effects  of  the  LP  coefficients  and  the  system 
filter  on  the  noise,  estimation  of  the  frequencies  is  recast  as  the  estimation  of  the  LP 
coefficients.  This  section  concludes  by  developing  a  bound  on  the  LP  coefficient  estimation 
error. 


3.4-1  Theoretical  Background.  To  show  the  relationship  between  sinusoidal  fre¬ 
quencies  and  a  set  of  LP  coefficients,  consider  a  signal  s[m],  consisting  of  the  sum  of  P 
real  sinusoids.  As  shown  in  Chapter  II,  there  exists  a  set  of  2P  -j-  1  LP  coefficients,  a[p], 
such  that  (8) 

2P 

a,\p]s[m  -  pj  =  0 

p=0 

Since  each  frequency  of  s[m]  is  a  zero  of  the  polynomial,  A{z),  formed  from  the  LP  co¬ 
efficients,  the  2P  frequencies  present  in  s(Tn]  can  be  found  as  roots  to  A{z).  The  search 
for  the  P  frequencies  comprising  the  sinusoids  in  s[m]  has  been  recast  as  the  search  for 
the  cissociated  2P  -I- 1  LP  coefficients.  As  shown  in  Chapter  II,  many  algorithms  based  on 
estimating  these  coefficients  have  been  developed  in  the  past.  This  dissertation  employs 
stochastic  modeling  techniques  to  derive  a  new  method  of  LP  modeling  which  is  based  on 
the  general  linear  system  model  (40:94-94) 

t  =  U<t>  +  v  (3.31) 
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In  this  model,  t  is  an  iVf-dimensional  vector  called  the  measurement  vector,  0  is  a  P- 
dimensional  vector  called  the  parameter  vector,  U  is  an  M  by  P  matrix  called  the  obser¬ 
vation  matrix  and  v  is  called  the  measurement  error  vector  (60:10). 

3.4- 2  Construction  of  the  LP  General  Linear  Model.  A  convenient  method 
of  calculating  the  LP  coeflBcients  can  be  found  by  modeling  Equation  2.14  in  terms  of 
the  general  linear  system  model  of  Equation  3.31.  Since  Equation  2.14  must  hold  for 
2P  <  m  <  M  —  1,  a  deterministic  linear  model,  parameterized  by  the  LP  coeflBcients  in 
the  vector  a  =  [a[0] . . .  a[2P]p,  can  be  constructed  as 

s[M-l]  ...  s[Af-l-2P]l[  a[0]  [o 
Sa=  I  ...  :  i  =  i  (3.32) 

s[2P]  ...  s[0]  a[2P|  0 

or  eqmvalently  Sa  —  0.  Here,  the  M  —  2P  by  2P  -I-  1  matrix  S  is  deterministic  with  a 
rank  of  2P  so  that  a  resides  in  the  null  space  of  5  (45).  Imposing  the  constraint,  o[0]  =  1 
yields  the  deterministic  general  linear  model 

s„  =  -S„ao  (3.33) 

Here,  Ug  =  [a[l] . . .  a[2P]]^,  while  Sg  is  the  first  column  of  S  and  Sg  is  the  observation 
matrix  of  the  remaining  columns  of  S.  The  vector,  Sg,  can  now  be  thought  of  as  being 
generated  from  the  deterministic  matrix,  —Sg,  so  that  only  4P  -t-  1  samples  are  needed 
to  determine  Oq  exactly.  Unfortunately,  sfm]  is  not  usually  available  and  estimates  of  Oo 
must  be  obtained  using  the  noisy  measurements  y\m]. 

3.4- 3  Noise  Effects  in  the  Measurements.  To  study  the  eiflfects  of  noise  in  the 
measurements,  assume  Sg  is  known  exactly  while  the  elements  in  Sg  are  corrupted  with 
additive  noise,  w[m],  so  that  3[Tn]  =  y[m\  —  w[m\  for  m  =  2P . . .  M  —  1.  In  this  case,  the 
form  of  the  LP  general  linear  model  becomes 

Vg  =  -Sgag  +Wg  =  Sg+Wg  (3.34) 
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Here,  Wg  can  be  viewed  ais  the  error  between  the  measurement  vector,  y„,  and  the  vector 
Sg  generated  by  the  LP  general  linear  model.  The  least  squares  estimate  for  Ug  which 
minimizes  the  sum  of  the  squared  errors  is  given  as  (60:31) 


a{LS)  =  -[SjSg]-'sJy,  (3.35) 

A  basic  assumption  for  this  solution  is  that  the  impact  of  the  error  in  each  mecisurement, 
y[Tn],  is  the  same  (83:203).  When  this  is  not  the  case,  a  weight  matrix,  C~^,  can  be  inserted 
to  provide  the  Weighted  Least  Squares  (WLS)  estimate  (60:31), (40:141). 

a{WLS)  =  -  SjC-^y,  (3.36) 

For  the  data  model  given  by  Equation  3.13,  the  noise  is  assiimed  to  be  zero  mejin  cind 
normally  distributed  with  covariance  matrix  Kw  Thus,  the  measurement  vector  is 
normally  distributed  with  the  following  properties 

Mean  =  my^  =  -SgUg  (3.37) 

Covariance  =  Ky^  =  (3.38) 

As  such,  the  ML  estimate  for  the  LP  coefficients  is  given  by  (60:152-155), (40:185-187) 

ag{ML)  =  -  SjK^\yg  (3.39) 

This  estimate  is  unbiased  with  a  covariance  matrix  given  by  (60:110-113) 

Ka,iML)  =  [SjK^\Sg]-'  (3.40) 

If  the  deterministic  observation  matrix,  5o,  weis  known,  the  ML  estimate  obtained  in 
Equation  3.39  would  represent  the  minimum  variance,  unbiased  estimate  obtainable  from 
the  mecisurements.  Unfortunately,  the  observation  matrix  must  also  be  estimated  from  the 
measurements. 
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3.4-4  ^oise  Effects  in  the  Observation  Matrix.  The  main  drawback  in  using  the 
LP  general  linear  model  to  identify  the  LP  coefficients  is  that  the  effects  of  the  noise  are 
embedded  in  the  observation  matrix.  That  is,  since  s[Tn]  =  i/[m]  —  the  observation 

matrix  itself,  Sg,  is  perturbed  by  the  noise.  In  terms  of  the  true  LP  coefficients,  the  LP 
general  linear  model  becomes 

Vo  =  -yoO-o  +  tCo  +  Wgag  (3.41) 

where  the  matrices  Yg  and  Wg  zire  defined  in  a  manner  similar  to  Sg.  Defining  the  error  eis 

2{a)  =  Wg +  Wgao  =  Wa  (3-42) 

the  perturbed  form  of  the  LP  general  linear  model  becomes 

Vg  =  -Ygag  +  z(o)  (3.43) 

The  error  vector,  z{a),  now  has  statistics  which  are  dependent  on  the  LP  coefficients  in 
addition  to  the  statistics  of  the  noise  vector  w.  To  show  this  dependency,  define  A  as  the 
M  hy  M  —  2P  matrix  of  LP  coefficients  so  that 

a[0]  ...  a[2F]  0  0  ...  0 

0  a[0]  ...  a[2P]  0  ...  0 

^  ^  ^  ^  (3.44) 

:  :  :  :  0  :  0 

_  0  0  ...  0  o[0]  ...  a[2P] 

This  matrix  has  a  rank  of  M  —  2P  and  allows  the  error  vector,  z{a)  to  be  written  eis 

z{a)  =  Wg +  Woao  =  Wa  =  A^w  (3.45) 

Since  m  is  a  zero-mean,  normally  distributed  noise  vector  with  covariance  Kw^  tlie  error 
vector  is  a  zero- mean,  normally  distributed  random  vector  with  a  covariance  matrix,  Kz(a)^ 
given  by 

Kz[a)  =  A^KwA  =  a^A^H'^HA  (3.46) 
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In  order  to  account  for  this  colorization  in  the  covariance  matrix,  let  G{a)  be  a  square 
M  -  2P  matrix  derived  from  the  Cholesky  decomposition  of  the  error  covariance  matrix 
(73:440-441). 

G^ia)G{a)  =  (3.47) 

Premutiplying  Equation  3.43  by  G{a)  yields  a  new  form  of  the  LP  general  linear  model 

G{a)y^  =  -G(a)Voaa  -t-  G(a)z(a)  =  -G(o)y;Oo  -H  e(a)  (3.48) 

Here,  e(a)  is  a  zero  mean,  normally  distributed  random  vector  with  covariance  matrix 
Ke{a)  =  I-  The  form  of  the  LP  general  linear  model  now  becomes 

G{a)Ya  =  e{a)  (3.49) 

The  estimation  of  the  P  frequencies  in  s[m]  from  Equation  3.13  has  been  recast  as  the 
estimation  of  the  2P  -I-  1  LP  coefficients  using  the  LP  general  linear  model  derived  in 
Equation  3.49.  The  number  of  LP  coefficients  to  be  estimated  can  be  reduced  further  by 
imposing  sinusoidal  constraints  on  the  coefficients. 

3.4-5  Imposition  of  Sinusoidal  Constraints.  To  properly  estimate  P  real  sinusoids 
using  the  order  polynomial,  A{z),  as  shown  in  Appendix  C,  the  LP  coefficients  must 
be  constrained  to  be  symmetric  about  F  so  that  a[p]  =  a[2F  —  p]  for  p  =  0 . . .  F  (8).  To 
impose  this  constraint,  let  a  =  [a[0], . . .  a(F]p  be  a  vector  containing  the  constrained  LP 
coefficients.  Thus,  a  is  related  to  a  via  the  linear  transformation  o  =  Bot.  Here  B  is  the 
constraint  matrix  defined  cis 

Ip  Op 

F  =  o?  1  (3.50) 

I  Bp  Op 
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and  Op  represents  a  vector  of  P  zeros,  Ip  represents  the  P  square  identity  matrix,  while 
I  Bp  represents  the  P  square  ‘Backward’  identity  matrix  given  by 


[IBph 


* 

1  for  j  =  P  +  1  —  r 

< 

0  otherwise 


(3.51) 


The  search  for  the  LP  coefficients,  contained  in  a,  of  a  2P‘'‘  order  model  has  been  reduced 
to  the  identification  of  the  P  +  1  constrained  coefficients  contained  in  Q.  Imposing  these 
constraints  on  the  LP  general  linear  model  yields 


G{a)YBa  =  Gia)Yca  =  e(a)  (3.52) 

Here,  Yc  is  an  iVf  —  2P  by  P  +  1  matrix  of  rank  P  +  1.  Now  let  be  the  first  column 
of  Yc  and  let  Yc  be  an  M  —  2P  by  P  matrix  of  the  remaining  columns  of  Yc-  Defining 
ycipi)  =  G(a)yj  and  Yc{ot)  =  G{a.)Yc  while  letting  olq  =  [a[l]  •  •  •  “[-P]]^»  alternate  form 
of  the  LP  general  linear  model  becomes 

Vcia)  =  -Yc{a)ao  +  e[a)  (3.53) 

Equation  3.53  represents  the  true  general  linear  model  relating  the  LP  coefficients  to  the 
frequencies  present  in  P  sinusoids  in  noise.  The  problem  now  is  to  estimate  the  LP  pa¬ 
rameters  in  Qo  and,  consequently  G(a),  given  the  M  observations.  Before  developing  ziny 
estimator  for  the  LP  coefficients,  it  is  useful  to  have  a  bound  on  the  accuracy  that  can  be 
obtained  from  the  mezisurements. 


3-4-6  Bounding  the  Estimation  Error.  In  general,  let  a  be  a  function  of  a  random 
vector  6  so  that  a  =  g{9).  It  can  be  shown  the  CRLB  for  a  can  be  derived  from  the  CRLB 
of  0  as  (40:45) 


Ka- 


-dgjd) 

dO 


[p(0)r‘ 


-dgje) 

d9 


>[0] 


(3.54) 
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Again,  ^{0)  is  the  Fisher  Information  matrix  for  6  while 


dg{d) 


ae 


99,{0) 


.o  96, 


(3.55) 


For  the  LP  coefficients,  the  functions  relating  the  P  unique  coefficients  to  be  estimated  to 
the  corresponding  frequencies  can  be  found  by  expanding  the  LP  polynomial 


^a[p]z  '’  =  ^  -  2cos(27r/p)2  ^  +  1] 

p=0  p=l 


(3.56) 


In  particular,  for  P  =  1 


a[l]  =  -2cos(27r/i)  =  g,{fi) 


(3.57) 


while  for  P  =  2 


a[l]  =  -2cos(27r/i)  -  2cos(2ir/2)  =  gi{f) 
a[2]  =  2  +  4cos(27r/i)cos(27r/2)  =  g2{f)  (3.58) 

Figure  3.6  shows  the  CRLB  for  the  LP  coefficients  for  one  and  two  sinusoids.  For  one 
sinusoid,  the  estimation  accuracy  will  decrease  as  the  sinusoidal  frequency  approaches  .25. 
This  is  to  be  expected  since  small  changes  in  the  frequency  around  /  =  .25  will  produce 
large  changes  in  the  value  of  the  LP  coefficient.  For  two  sinusoids,  as  shown  by  the  contour 
plot  of  Figure  3.6b,  the  estimation  accuracy  of  the  LP  coefficients  is  primarily  a  function  of 
the  frequency  separation  of  the  sinusoids.  Consequently,  the  estimation  accuracy  decreases 
dramatically  for  |/i  —  /2I  <  .02.  This  is  to  be  expected  since  the  CRLB  for  the  frequencies 
follows  the  same  gener2il  form  as  the  CRLB  of  the  coefficients. 

3.4-7  Section  Summary.  This  section  derived  the  true  LP  general  linezir  model 
for  estimating  the  set  of  P  imique  LP  coefficients  generating  P  sinusoids  in  noise  and  is  an 
original  contribution  of  this  research.  Specifically,  by  properly  incorporating  the  effects  of 
both  the  filter  and  the  LP  coefficients  upon  the  noise,  the  LP  general  linear  model  derived 
in  Equation  3.53  represents  the  true  linear  model  for  estimating  the  LP  coefficients.  Any 
other  model  relating  the  estimation  of  the  LP  coefficients  to  the  sinusoidal  frequencies  will 
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CRLB  For  LP  Coefficient 


(a)  Sinusoid  Parameters:  [6°  =  l,(^i  =  7r/3],  Filter  Parameters: 
Center  Frequency:  fc  =  .21;  Bamdwidth:  /b  =  .2  Length:  N  =  32. 


Combined  CRLB  for  Two  LP  Coefficients 


(b)  Sinusoid  Parameters:  [6°  =  1,(^J  =  tr/3],  [63  =  1,02  =  47r/3]. 

Figtire  3.6  CRLB  for  LP  CoeflBcients:  Variance  bounded  as  function  of  frequency  at 
fixed  SNR  of  lOdB  and  block  length  M  =  32.  For  two  LP  coefficients,  contours  found 
asVar  =  .5{Far{o[l]}  +  Var{a[2]})  and  plotted  as  - 10  logio( Var).  SNR  calculated 
as  SNR  =  -101ogio(2CT^). 
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provide  suboptimal  frequency  estimates.  Estimating  the  frequencies  has  been  reccist  as 
estimating  the  coefficients  of  the  LP  general  linear  model.  This  derivation  naturally  leads 
to  the  employment  of  a  Fixed  Point  concept  to  determine  the  LP  coefficients.  This  is  one 
of  the  main  original  contributions  of  this  dissertation. 

3.5  LP  Coefficient  Estimation:  Fixed  Point  Concept 

This  section  derives  the  estimators  to  be  used  for  estimating  the  LP  coefficients  as 
defined  by  the  LP  general  linear  model  derived  in  Equation  3.53.  By  viewing  the  optimal 
estimates  eis  fixed  points  in  the  domain  of  allowable  solutions,  this  section  develops  two 
alternate  methods  of  estimating  the  LP  coefficients.  One  estimator  is  based  on  an  Iterative 
Least  Squares  (ILS)  technique  and  the  other  on  an  Iterative  Total  Least  Squares  (ITLS) 
technique.  Both  of  these  estimators  provide  accurate  estimates  of  the  LP  coefficients  over 
a  wide  range  of  SNRs  and  represent  original  contributions  in  the  area  of  LP  parameter 
estimation. 

3.5.1  Theoretical  Background.  The  technique  to  be  used  for  estimating  the  LP 
coefficients  from  the  general  linear  model  of  Equation  3.53  is  based  on  the  theory  of  fixed 
points  and  reviewed  in  Appendix  D.  To  apply  this  theory  to  the  estimation  process,  a 
nonempty  set,  S,  must  be  defined  along  with  a  function,  d,  which  computes  the  distance 
between  any  two  members  of  3  (33:1).  Assuming  the  properties  of  a  general  distance 
function  hold,  then  the  pair  [5,  d]  is  called  a  metric  space  (58:21).  Now  let  £  :  5  — >  5  be 
a  function  from  a  metric  space  [<S,  d]  into  itself.  A  point  q  €  «S  is  cedled  a  fixed  point  of  £ 
provided  (2:92) 

£(q)  =  a  (3.59) 

In  general,  there  are  many  requirements  which  must  be  satisfied  for  a  fixed  point  of  a 
function  to  exist  (33).  However,  the  following  two  theorems  will  guarantee  the  existence  of 
a  fixed  point  of  an  arbitrary  nonlinear  mapping,  £,  provided  the  metric  space  is  adequately 
defined: 
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•  Schauder’s  Fixed  Point  Theorem  -  Let  £  be  a  mapping  on  a  Banach  space,  S  such 
that  £  :  S  S  is  continuous  and  bounded  and  the  image  of  each  bounded  set  hcis  a 
compact  closure.  Then  there  exists  in  5  a  fixed  point  a  of  £  (33:152). 

•  Brouwer’s  Fixed  Point  Theorem-  Let  £  be  a  continuous  mapping  on  Si{0, 1),  the  unit 
sphere  in  L-dimensional  Euclidean  space.  If  £(Q()  £  5^(0,!)  for  all  O;  €  5^(0,!), 
there  exists  a  fixed  point  of  £,  denoted  a  in  1)  (33:116). 

In  this  dissertation,  the  metric  space  to  be  employed  for  estimating  the  LP  coefficients  will 
be  subsets  of  the  general  L-dimensional  Euclidean  space. 


3.5.2  Metric  Space  Definition.  Define  <5^(0,  k)  as  the  set  of  X-dimensional  vectors, 

a  =  [oi . . .  with  ai  €  R,  such  that  the  two-norm  of  a  is  finite. 

L 

||a||^  =  a^a  =  =  A:  <  oo  (3.60) 

j=i 


This  implies  each  component  of  a  is  finite.  Thus,  5i(0,  fc)  is  a  closed  and  boimded  set 
of  R^,  the  Euclidean  L  space  (58).  Now  define  a  distance  function  ,  d{ai,aj)  for  any 


€  5^(0,  A:)  as 


d{ai,aj) 


N  <=i 


(3.61) 


The  pair,  [5^(0,  fc),  d],  is  a  metric  space.  In  addition,  under  this  distance  metric,  is  a 
Banach  space  (58:112)  and  Si{0,k)  is  called  compact  (58:62).  Provided  a  function  £  can 
be  constructed  which  maps  €  Si{0,k)  into  ocj  €  Si{0,k),  then  a  fixed  point  of  £  will 
exist.  The  functions  to  be  constructed  will  be  based  on  the  LS  and  TLS  solutions  to  the 
over  determined  system  of  equations  given  by 


Ycia)a„  =  -y,{a) 


(3.62) 


3.5.3  Iterative  Least  Squares  Fixed  Point  Mapping. 

3.5.3. 1  Theory.  In  general,  the  LS  solution  to  Equation  3.62  consists  of 
finding  the  set  of  coefficients,  q^,  such  that  the  sum  of  the  squared  terms  of  the  error 
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vector,  e(Qo),  defined  by 


e(Q„)  =  V,(a„)a^  +  y,(a^) 


(3.63) 


is  minimized  (83:154).  Thus,  the  optimal  solution  is  given  as 

do  =  arg(min{e(ao)^e(ao)})  (3.64) 

This  is  the  same  as  locating  the  point  y<.(o(o)  in  column  space  of  Vdao)  which  is 
closer  to  J/<.(ao)  than  any  other  point  (83:154).  Thus,  e(Qo)  must  be  perpendicular  to  the 
columns  of  ^(Qio)  so  that  T/ (ao)e(ao)  =  0  (87:524).  Substitution  yields  the  least  squares 
estimate  derived  from  the  observations  as 

dio  =  -  [V'/(ao)y;(a<,)]~^>'/(ao)y(Q:o)  =  £ls(Oo)  (3.65) 

This  is  equivalent  to  finding  the  error  vector,  e(Qo),  of  minimum  norm  which  must  be 
added  to  the  vector,  to  bring  it  into  the  range  of  Yc{olo)  (30),  (67).  To  use  this 

function  as  an  estimator  of  the  LP  coefficients,  a  fixed  point  must  exist. 

3. 5. 3. 2  Existence.  To  show  a  fixed  point  exists  for  the  LS  mapping  function, 
£is,  let  Of  6  5/j(0,  fc).  The  LS  mapping  of  aj  yields 

a,  =  £,s(a,)  =  -  [y7G^(a,)G(a,)y;]''  [r/G^(a,)G(Q,)yJ  (3.66) 

To  show  otj  €  Sp{0,  k),  rewrite  G(Q{)  in  terms  of  its  SVD  as 

G(a,)  =  (3.67) 

where  U  =  [ui  ...UAr_2i>],  V  =  [ui  ...Vm-2p\  are  square  unitary  matrices  and  S  is  an 
M  —  2P  square  diagonal  matrix  containing  the  singular  values  of  G(Q()  arranged  so  that 
(Tj  >  aj  for  i  >  j.  Now,  where  Aj  is  an  eigenvalue  of  the  nonsingular  matrix 
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AjKwAi.  Since,  A*  >  0  for  i  =  1 . . .  M  -  2P,  the  vector,  x  =  G{at)y^,  can  be  written  as 

M-2P 

Y.  (3.68) 

m=l 

Here,  x^x  <  oo  so  that  x  €  Sp{0,k).  Now,  assuming  Yc  is  of  rank  P,  then  the  matrix 
product,  Yc{oti)  =  YcG{qi)  is  also  of  rank  P.  Decomposing  via  the  SVD  yields 

y^Ca,)  =  EDQ^  (3.69) 

where  E  =  [ci  ...eM-2p]  Q  —  [Q\  ■■■Qp]  «^re  square  unitciry  matrices  and  D  is  an 
M  —  2P  hy  P  diagonal  matrix  containing  the  nonzero  singular  values,  dp,  for  p  =  1 . . .  P, 
ofy,(a,).  Thus 

||£l5(«()II^  =  OL^OLj  =  ®  ^  <  oo  (3-70) 

p=i  p=i 

and  for  any  ai  €  5/>(0,  fc),  then  aj  =  £1,5(01)  €  Sp{Q,k). 

To  show  £is  is  continuous,  define  the  matrix  Rp  as  the  M  hy  M  —  2P  matrix  with 

1  for  i  =  j '  +  p 

~  1  for  i  =  2P  -  p  +  j  (3.71) 

0  otherwise 

for  i  =  1 . . .  Af  and  j  =  I . . .  M  -  2P.  Thus,  for  p  =  0 ...  F,  i?p  is  of  rank  M  -  2P  and  so 
is  the  sum  of  any  Po  such  matrices.  The  matrix  A  C2in  now  be  decomposed  as  the  sum  of 
these  matrices  as 

p 

A  =  R,  +  Y  «b]^  =  -R.  +  R(a)  (3.72) 

P=1 

Using  this  decomposition,  the  M  —  2P  square  matrix  A^KyjA  can  be  written  as  a  rational 
fimction  of  a  cis 

A^KwA  =  [Ro  +  i?(Q)]^  Kw  [Rp  +  R(a)]  (3.73) 
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Because  sums,  products  and  inverses  of  rational  functions  are  also  rational  functions,  then 
■^Ls(o:)  is  a  rational  function  of  oc  (tO).  Therefore,  ££,5  is  continuous  everywhere  it  is 
defined.  Now,  Kyj  A  is  invertible  and  of  rank  M  -  2P.  Since  is  assumed  to  be  of 
rank  P,  then  the  matrix  product  Yj[A'^ KwA\-'^Yc  is  of  rank  P  so  that  its  inverse  exists. 
Thus  Lis{ol)  is  defined  for  all  a  and,  as  such,  is  continuous  on  iSp(0,  fc)(48;857). 

Finally,  to  show  ££,5  is  bounded,  the  limit  as  q  grows  unbounded  must  exist  and  be 
bounded  (70).  First,  define  the  set 

<Sp(0,l)  =  {x€P"’:||x||  =  l}  (3.74) 

and  let  a  —  0x  for  P  €  R.  Then 

£w(/Sx)  =  -jy/  Kw  [j  +  Ri^))  n| 

.y/  +  R(^f\  Ky,  +  R{x)j  y,  (3,75) 

Taking  the  limit  as  /?  approaches  infinity  yields 

£is(/3x)  =  -  (y/[P^(x)ii:«,P(x)]-iy,)"'  Yj[R^ix)KwR{x)]-^y,  =  £^5,^(2:) 

(3.76) 

Since  /?(x)  is  still  of  rank  M  —  2P,  then  £-ls.oc{0x)  is  simply  the  solution  to  the  weighted 
least  squares  problem  and  is  therefore  a  finite  vector  with  ||£Ls,x(/?2:)ir  <  Finally,  let 

As  shown  in  (70),  /(/3)  is  a  continuous  function  with 

lim /(/?)=  max  Lis.^ix)  <  00  (3.78) 

X£Sp{0.k)  ^  ' 
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Since  /(/3)  is  a  continuous  function  bounded  at  infinity,  so  is  its  norm  and  the  supremum 
of  f{P)  is  finite.  That  is  (70) 


sup|l££,s(a)ll  =  sup||/(,d)||  <  oo  (3.79) 

Q  a 

Thus,  the  supremum  of  the  infinity  norm  of  £^5  is  finite  and  maps  the  bounded  set 
Sp{0,  k)  C  into  itself.  Finally,  since  £^5  is  a  continuous  function  that  maps  a  convex 
compact  set  into  a  convex  compact  set,  then  via  Schauder’s  Theorem  (33:152-153),  there 
exists  a  fixed  point  such  that  a  6  Sp{0,  k)  such  that  £^5(0)  =  a.  The  fixed  point  obtained 
using  an  iterative  form  of  the  LS  function  of  Equation  3.65  will  become  the  estimate  of 
the  LP  coeflBcients. 


3. 5. 3. 3  Application.  Prom  the  true  system  model,  where  a  indicates  the 
true  LP  coefficients, 


y^{ao)  =  -YcMao  +  e(Qo) 


(3.80) 


Clearly,  the  quantity  -y^{ao)  +  e(ao)  is  in  the  range  of  Ydao)-  If  the  least  squtires 
estimate,  dco  is  given  by  Equation  3.65,  substitution  yields 


a„=  [y/(Q„)y,(Q<,)] 


-1 


K  (Oo)  [yc(ao)ao  -  e(Qo)] 


(3.81) 


For  do  to  be  equal  to  Oq,  the  error  vector  must  be  orthogonal  to  the  columns  of  Ydcto)- 
In  this  case,  the  true  values  of  the  LP  coefficients,  do,  would  be  a  fixed  point  of  the  Least 
Squares  mapping 

Oo  =  £is(ao)  (3.82) 

Now  consider  the  iterative  form  of  the  LS  mapping  function  Lns-  From  the  system  model, 
at  iteration  i, 

-  VM)  +  e(Oo)  =  ■5^c(o£o)Oo  (3-83) 

The  Iterative  Least  Squares  (ILS)  estimate  for  the  LP  coefficients  is  found  from  Equation 
3.65  as 

=  -  [yo^(a‘)yo(a‘)]"^  n’'(o‘)l/(a‘)  =  £/L5(a‘)  (3.84) 
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At  each  iteration,  the  squared  error,  calculated  as 


Jiai)  =  e\a.'Ma\)  (3.85) 

is  minimized  with  respect  to  the  current  LP  coeflBcient  estimate.  When  =  aj,,  the 
error  vector,  e(a‘)  is  orthogonal  to  the  columns  of  K(o!’)  and  q  0  =  £/ts(ao)-  A  fixed 
point  has  been  reached  and  the  estimate  of  the  LP  coefficients  becomes  ctg  =  a^. 

3. 5. 3. 4  Simulations.  Figure  3.7  shows  a  typical  convergence  of  the  ILS 
algorithm  for  one  and  two  filtered  sinusoids.  In  both  cases,  the  center  fi’equency  of  the  filter 
W81S  used  to  provide  the  initial  LP  estimates  and  less  then  ten  iterations  were  required  for 
location  of  the  fixed  point.  In  addition,  the  fixed  point  located  does  not  coincide  with  the 
LP  coefficients  given  by  the  minimum  of  the  error  surf2ice.  Figure  3.8  shows  the  estimation 
accuracy  of  the  ILS  fixed  point  mapping  for  a  single  sinusoid  as  a  function  of  SNR  and 
frequency.  FigTire  3.8a  shows  the  ILS  algorithm  produces  accurate  frequency  estimates 
only  when  the  LP  general  linear  model  incorporates  the  effects  of  the  filter  upon  the 
noise.  Assumptions  of  uncorrelated  (white)  noise  produce  an  incorrect  model.  In  addition, 
provided  the  correct  model  is  used,  the  accuracy  of  the  estimator  is  relatively  independent 
of  the  method  of  initialization  above  the  SNR  of  about  3dB.  For  lower  SNRs,  the  algorithm 
becomes  sensitive  to  the  initial  LP  estimates  and  the  threshold  can  be  extended  to  about 
OdB  using  the  filter  center  frequency  to  provide  initial  estimates.  Figme  3.8b,  shows  the 
frequency  estimation  accuracy  of  the  ILS  fixed  point  estimator  across  the  entire  range  of 
frequencies  for  a  sinusoid  in  zero-mean,  normally  distributed,  uncorrelated  noise.  Using  the 
simple  LS  estimate  as  the  initial  estimate,  this  figure  shows  the  ILS  fixed  point  algorithm 
provides  optimal  frequency  estimates  except  for  extremely  low  (/  <  .01)  or  high  (/  >  .49). 
frequencies. 

Figure  3.9a  shows  the  accuracy  of  the  ILS  fixed  point  estimator  for  the  two  widely 
spaced  sinusoids  in  zero-mean,  independent,  normally  distributed  noise.  Using  an  initial 
estimate  obtained  from  the  general  LS  solution,  this  figure  shows  the  frequency  estimates 
obtained  as  a  result  of  the  ILS  estimator  are  near  optimal  for  SNRs  above  threshold.  As 
with  a  single  sinusoid.  Figure  3.9b  shows  the  ILS  algorithm  produces  accurate  frequency 
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estimates  only  when  the  LP  general  linear  model  incorporates  the  effects  of  the  filter  upon 
the  noise.  Assumptions  of  uncorrelated  noise  produce  an  incorrect  model  and,  conse¬ 
quently,  suboptimal  estimates.  For  two  sinusoids  however,  initializing  the  algorithm  with 
LP  coefiBcients  obtained  near  the  center  filter  center  frequency  produced  accurate  estimates 
at  a  slightly  lower  SNRs  than  those  produced  by  a  simple  LS  initial  estimate. 

The  ability  to  estimate  the  frequencies  of  both  widely  and  closely  separated  sinusoids, 
either  in  independent  or  correlated,  normally  distributed  noise,  imply  the  ILS  estimator 
can  be  used  as  a  general  frequency  estimator. 


3.5.4  Iterative  Total  Least  Squares  Fixed  Point  Mapping. 


3. 5.4.1  Theory.  In  forming  the  LS  solution,  the  noise  in  the  observation 
matrix,  Yc{cXo),  was  ignored.  The  LS  solution  simply  found  the  vector  of  minimum  norm 
which  must  be  added  to  the  vector  —Vcioco)  to  bring  it  into  the  range  of  y’(.(Q:o)-  The  TLS 
solution  attempts  to  account  for  the  noise  in  the  observation  matrix,  Ydoco)  in  addition  to 
the  noise  in  the  yd^o)  (21), (67), (87:533-538).  First,  Equation  3.62  can  be  rewritten  as 


[j/,(Q)|ye(a)l 


1 

Oto 


=  0  =  yc(«)Q: 


(3.86) 


The  TLS  solution  for  q  seeks  the  matrix,  E{cc),  of  minimum  Frobenius  norm  such  that 
the  matrix  sum,  [Vc(of)  +  ^(<3e)],  is  rank  deficient  (21),  (67).  Decomposing  Yc{a)  via  the 
SVD  yields 

Ycia)  =  U{a)Eia)V^{a)  (3.87) 

Here,  U{a)  and  V{a)  are  unitary  matrices  which  can  be  written  in  terms  of  their  column 
vectors  as 


U{a)  =  [«i(q:)  . . .  Um-2p{oc)]  with  Ui  6 
V{a)  =  [vi (q) ...V p+i (q)]  with  Uj  G 


(3.88) 

(3.89) 
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Fixed  Point  Convergence  of  ILS  Algorithm 


(a)  Sinusoid  Parameters;  [6°  =  l,/i  =  .192,  =  tt/S]  and  SNR  —  -5dB. 

Final  Objective  Function  Evaluation:  Jictus)  —  37.26,  Jmin  =  35.009. 


Fixed  Point  Convergence  of  the  ILS  Algorithm 


(b)  Sinusoid  Parameters:  [6°  =  l,/i  =  .207, =  7r/3],  [62  =  l,/2  =  .227,02  =  47r/3]. 
Objective  Function  Evaluation:  J(aiis)  =  3.6088,  Jmin  =  3.5230;  SNR  =  5  dB. 

Figure  3.7  ILS  Fixed  Pont  Convergence:  J{ao)  calculated  at  fixed  SNR  and 

block  length  M  =  32.  Ten  iterations  allowed  for  convergence.  Filter  Parameters: 
Center  Frequency:  fc  =  .21;  Bandwidth;  fs  =  .2;  Length:  N  =  32.  SNR  calculated 
as  SNR  =  -101ogio(2cr^). 
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Frequency  Measurement  Accuracy 


(a)  Sinusoid  Parameters:  [6°  =  l,fi  —  .192, =  ir/3],  MSE  at  .5dB  SNR  intervals. 
Filter  Pajameters:Center  Frequency:  fc  =  .21;  Bandwidth:  /b  =  -2;  Length:  N  =  32. 


Frequency  Measurement  Accuracy 


(b)  Sinusoid  Paraimeters:  [b°  =  1,0J  =  7r/3],  SNR  =  lOdB. 
MSE  at  .005  frequency  intervals. 


Figure  3.8  ILS  Frequency  Estimation  Accuracy-One  Sinusoid:  MSE  calcu¬ 

lated  from  500  independent  realizations  of  block  length  M  =  32.  Ten  itera¬ 
tions  allowed  for  convergence.  The  incorrect  model  assumed  Kw  =  while  the 
correct  model  used  SNR  calculated  as  5N'/?  = -101ogio(2<T^); 

A/5F=5^Ei=i{/iW-/i)- 
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Combined  Frequency  Measurement  Accuracy 


Combined  Frequency  Measurement  Accuracy 


(b)  Sinusoid  Parameters:  [6°  =  l,/i  =  —  tt/S],  [b^  =  l,/2  =  -227, (j>2  =  4;r/3]. 

Filter  Parameters:  Center  Frequency:  fe  =  .21;  Bandwidth:  /b  =  .2  Length:  N  =  32. 


Figure  3.9  ILS  Frequency  Estimation  Accuracy- Two  Sinusoids:  MSB  calcu¬ 

lated  at  SNR  intervals  of  .5dB  from  500  independent  realizations  of  Af  =  32  sam¬ 
ples  of  sinusoids  in  noise.  The  incorrect  model  assumed  Kw  =  cr^  I  while  the 
correct  model  used  SNR  calculated  as  SN'i?  = -10  logio(2CT^); 
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In  addition,  E(a)  is  an  M  —  2P  by  P  +  1  diagonal  matrix  of  the  singular  values  of  lc(ck)i 
denoted  CTi  for  i  =  1 . . .  P  +  1,  arranged  so  that  (Tj  >  ctj  . . .  >  crp+j  >  0.  The  TLS  solution 
for  Q  is  given  by  the  mapping  function,  £tls,  as  (86:534) 


a  = 


t?p+i(a) 


—  ■£tz,s(q:) 


(3.90) 


Provided  V^(a)i,P4.i  #  0,  this  solution  is  unique  and  provides  the  set  of  LP  coefficients 
aissociated  with  the  matrix  of  minimum  Frobenius  norm,  E{a)  such  that  (21) 


-  i/^(a)  +  e(a)  6  Range{Y^{a)  +  P(a)}  (3.91) 


In  order  to  use  this  estimation  technique  for  the  LP  coefficients,  a  fixed  point  must  exist. 


3.5.4  s  Existence.  To  show  a  fixed  point  exists  for  the  TLS  mapping 
function,  let  5p+i(0,l)  :  {a  G  R^^^\oJoi  =  1}  and  let  a,  €  Sp+i{0,l).  The  solution 
provided  by  £tls(o:i)  is  the  right-most  singular  vector 


/V.  = 


(3.92) 


Here,  V"(q()i  is  the  first  element  of  the  P  -h  1  column  of  V{a{).  Since  this  solution 
can  be  scaled  by  any  constant  without  changing  the  zeros  of  the  LP  polynomial  A{z),  a 
modified  form  of  the  TLS  mapping  becomes 


=  wp+iCotj)  =  £tls{oh)  (3.93) 

Since  Vp+i(Q,)^np+i(a,)  =  1,  then  a,  =  LtisCQi)  e  5p+i(0,l). 

Now,  to  show  £tls  is  continuous,  the  TLS  solution  was  found  via  the  SVD  of  the 
matrix  product  G(Q:)yc7.  Since  G{a)  is  a  square  M  —  2P  nonsingulaj:  matrix  and  Yc  is 
and  M  -  2P  by  P  -I- 1  matrix  of  rank  P  +  1,  then  Vp+i(q)  exists  for  all  a.  Furthermore, 
this  solution  is  the  same  solution  as  the  TLS  solution  of 


Yj[AjKwA,]~Wca  =  0 


(3.94) 
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This  function  is  a  rational  function  of  q,,  and  since  sums,  products  and  inverses  of  rational 
functions  are  also  rational  functions,  then  £.TLs{oi)  is  continuous  everywhere  in  its  domain 
(48:857).  Since  the  TLS  solution  exists  for  all  a  €  5p4.i(0, 1),  then  £tls(Q:)  is  continuous 
on  5p+i(0, 1).  Thus,  via  Brouwer’s  Fixed  Point  theorem,  there  exists  an  Q  G  5p+i(0, 1) 
such  that  a  fixed  point  exists  (33:116).  The  fixed  point  obtained  via  an  iterative  form  of 
the  TLS  solution  will  become  the  estimate  of  the  LP  coefficients. 


3. 5.4-3  Application.  From  the  true  system  model,  where  a  indicates  the 
true  LP  coefficients 

Y^{a)ao  =  -yc(«)  +  e(a)  (3.95) 

Again,  the  vector  —ydot)  +  ^(cc)  is  in  the  range  of  Kc(a).  Rearranging  terms  gives 

[Vc(a)-W'c(a)la  =  0  (3.96) 


Decomposing  Ycio.)  as  the  SVD  yields  the  TLS  estimate  as 


t>p+i(a) 

^(o')i,p+i 


(3.97) 


For  Q  to  be  equal  to  a,  the  matrix,  — lFc(o!))  must  be  of  minimum  Frobenius  norm.  In 
this  case,  the  true  values  of  the  LP  coefficients  become  a  fixed  point  of  the  TLS  function 


a  —  £TLs(ot)  (3.98) 

Now  consider  the  iterative  form  of  the  TLS  solution.  Prom  the  system  model,  at  iteration 
i,  the  model  yields 

yc7(a‘)Q‘  =  e(Q‘)  (3.99) 

The  Iterative  Total  Least  Squares  (ITLS)  estimate  for  the  LP  coefficients  is  foimd  via  the 
mapping  function,  £itlsi  as 

=  £m5(a‘)  (3.100) 
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At  each  iteration,  the  squared  error  is  calculated  as 


J(q)  =  e^(a‘)e(a')  (3.101) 

When  =  a‘,  the  noise  matrix,  W(a*)  is  of  minimum  Frobenius  norm  and  the  fixed 
point  has  been  reached  so  that  q‘  =  £rTLs(oi')- 

3. 5. 4-4  Simulations.  Figure  3.10  shows  a  typical  convergence  of  the  ITLS 
algorithm  for  one  and  two  filtered  sinusoids.  In  both  cases,  the  center  frequency  of  the  filter 
was  used  to  provide  the  initial  LP  estimates  and  less  then  ten  iterations  were  required  for 
location  of  the  fixed  point.  In  addition,  the  fixed  point  located  does  not  coincide  with  the 
LP  coefficients  given  by  the  minimum  of  the  error  surface.  Figure  3.11  shows  the  estimation 
accuracy  of  the  ITLS  fixed  point  mapping  for  a  single  sinusoid  as  a  function  of  SNR  and 
frequency.  Figure  3.11a  shows  the  ITLS  algorithm  produces  accurate  frequency  estimates 
only  when  the  LP  general  linear  model  incorporates  the  effects  of  the  filter  upon  the 
noise.  Asstimptions  of  uncorrelated  (white)  noise  produce  an  incorrect  model.  In  addition, 
provided  the  correct  model  is  used,  the  accuracy  of  the  estimator  is  relatively  independent 
of  the  method  of  initialization  above  the  SNR  of  about  3dB.  For  lower  SNRs,  the  algorithm 
becomes  sensitive  to  the  initial  LP  estimates  and  the  threshold  can  be  extended  to  about 
OdB  using  the  filter  center  frequency  to  provide  initial  estimates.  Figmre  3.11b,  shows  the 
frequency  estimation  accuracy  of  the  ITLS  fixed  point  estimator  across  the  entire  range 
of  frequencies  for  a  single  sinusoid  in  zero  mean,  normally  distributed,  uncorrelated  noise. 
Using  the  simple  LS  estimate  as  the  initial  estimate,  this  figme  shows  the  ITLS  fixed  point 
algorithm  can  provide  optimal  frequency  estimates  except  for  extremely  low  (/  <  .01)  or 
high  (/  >  .49).  frequencies. 

Figure  3.12a  shows  the  accmacy  of  the  ITLS  fixed  point  estimator  for  the  two  widely- 
spaced  sinusoids  in  zero-mean,  independent,  normally  distributed  noise.  Using  an  initial 
estimate  obtained  from  the  general  TLS  solution,  this  figure  shows  the  frequency  estimates 
obtained  as  a  result  of  the  ITLS  estimator  are  near  optimal  for  SNRs  above  threshold. 

As  for  a  single  sinusoid.  Figure  3.12b  shows  the  ITLS  algorithm  produces  accurate 
frequency  estimates  only  when  the  LP  general  linear  model  incorporates  the  effects  of  the 
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filter  upon  the  noise.  Assumptions  of  uncorrelated  noise  produce  an  incorrect  model  and, 
consequently,  suboptimal  estimates.  For  two  sinusoids  however,  initializing  the  algorithm 
with  LP  coefficients  obtained  near  the  filter  center  frequency  produced  accurate  estimates 
at  a  slightly  lower  SNRs  than  those  produced  by  a  simple  TLS  initial  estimate. 

The  ability  to  estimate  the  frequencies  of  both  widely  and  closely  separated  sinusoids, 
either  in  independent  or  correlated,  normally  distributed  noise,  imply  the  ITLS  estimator 
can  be  used  as  a  general  frequency  estimator.  However,  this  estimator  requires  an  SVD  of 
the  M  —  2P  by  P  +  1  observation  matrix  at  each  iteration. 

3.5.5  Section  Summary.  In  this  section,  two  methods,  based  on  fixed  point 
theory,  were  derived  for  estimating  the  coefficients  of  the  LP  general  linear  model.  One 
method,  termed  the  ILS  algorithm,  is  based  to  an  iterative  least  squares  solution  on  an 
over-determined  system  of  equations  while  the  other  method,  coined  the  ITLS  algorithm, 
is  based  on  an  iterative  total  least  squares  solution.  Simulations  indicate  both  methods 
provide  minimum  variance  imbiased  estimates  of  the  LP  coefficients,  and  consequently,  the 
sinusoidal  frequencies,  over  a  wide  range  of  SNRs.  All  other  factors  being  equal,  the  ILS 
method  would  be  preferred  over  the  ITLS  method  since  the  ITLS  algorithm  requires  an 
SVD  of  a  Af  -  2P  by  P  -I-  1  matrix  at  each  iteration.  Application  of  fixed  point  theory 
to  the  estimation  of  the  LP  coefficients  and  development  of  the  ILS  and  ITLS  fixed  point 
mapping  functions  is  an  original  contribution  of  this  research.  The  next  section  relates  the 
estimates  found  by  these  methods  to  the  ML  estimates  of  the  LP  coefficients. 

3.6  LP  Modeling/  ML  Frequency  Estimation  Relationship 

This  section  derives  the  exaurt  relationship  between  the  true  LP  general  linear  model 
and  ML  Frequency  estimation.  Specifically,  this  section  will  show  the  LP  coefficients, 
defined  by  the  general  linear  model  of  Equation  3.53,  will  provide  ML  frequency  estimates 
for  filtered  data  provided  the  two-norm  of  the  model  error  is  minimized.  This  derivation, 
establishing  an  exact  relationship  between  ML  frequency  estimation  of  filtered  sinusoids 
and  the  LP  general  linear  model,  is  an  original  contribution  of  this  dissertation. 
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Fixed  Point  Convergence  of  ITLS  Algorithm 


(a)  Sinusoid  Parameters:  (6i  =  l,/i  =  .192,  =  ir/3],  and  SNR  =  -5dB. 

Objective  Function  Evaluation:  J{cxitls)  =  38.71,  Jmin  =  35.009. 


Fixed  Point  Convergence  o1  ITLS  Algorithm 


(b)  Sinusoid  Parameters:  =  l,/i  =  .207,0"  =  7r/3],  [b^  =  l,/2  =  -227,02  =  47r/3]. 

Objective  Function  Evaluation;  J{aiTLs)  =  3.6883,  Jmin  =  3.5230;  SNR  =  5dB. 

Figure  3.10  ITLS  Fixed  Pont  Convergence;  J{a)  calculated  as  function  of  LP  coef¬ 
ficients  at  fixed  SNR  and  block  length  M  =  32.  Ten  iterations  allowed  for  con¬ 
vergence.  Filter  Parameters:  Center  Frequency:  fc  =  .21;  Bandwidth:  /a  =  .2; 
Length:  N  =  32.  SNR  calculated  as  SNR  =  -101ogio(2a^). 
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Frequency  Measurement  Accuracy 


(a)  Sinusoid  Parameters:  [6°  =  l,/i  =  .192, =  7r/3],  MSE  at  .5dB  SNR  intervals. 
Filter  Parameters:  Center  Frequency:  =  .21;  Bandwidth:  fa  =  -2;  Length:  N  =  32. 


Frequency  Measurement  Accuracy 


MSE  at  .005  frequency  intervals. 

Figrire  3.11  ITLS  Frequency  Estimation  Accuracy-One  Sinusoid;  MSE  c£d- 
culated  from  500  independent  realizations  of  block  length  M  =  32.  Ten  itera¬ 
tions  allowed  for  convergence.  The  incorrect  model  assumed  Kw  =  while  the 
correct  model  used  Kw  SNR  calculated  as  SNR=  -101ogio(2(T^); 
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Combined  Frequency  Measurement  Accuracy 


Combined  Frequency  Measurement  Accuracy 


(b)  Sinusoid  Paurameters:  [b°  =  l,/i  =  .207, d>i  =  7r/3],  [62  =  l,/2  =  -227,02  =  47r/3]. 
Filter  Parameters:  Center  Frequency:  fc  =  .21;  Bandwidth:  /b  =  .2;  Length:  N  =  32. 


Figure  3.12  ITLS  Frequency  Estimation  Accuracy- Two  Sinusoids:  MSE  cal¬ 
culated  at  SNR  intervals  of  .5dB  from  500  independent  realizations  of  M  =  32 
samples  of  sinusoids  in  noise.  The  incorrect  model  assumed  Kw  =  cr^I  while  the 
correct  model  used  Kw  =  cr^H^H.  SNR  calculated  as  SNR=  -101ogio(2cr^); 

mse  =  eJLi  E -=1(4(0  -  fp?- 


3-42 


3.6.1  ML  Objective  Function  Redefined.  From  Equation  3.25,  to  provide  ML 
frequency  estimates,  the  objective  function  J{f)  must  be  minimized  with  respect  to  /. 

Jif)  = 

To  minimize  this  ML  frequency  objective  function  ,  consider  a  vector  space  interpretation 
of  the  minimization  problem  and  assume  that  y  has  length  M  >  2P.  From  Equation  3.13, 
since  A  is  an  M  by  2P  matrix  with  2P  linearly  independent  columns,  the  column  vectors 
of  A,  denoted 

A  =  (Ai,  Aj, . . .  A2p]  (3.102) 

span  a  2P  dimensional  subspace,  Vj,  of  the  real  vectors,  .  As  such,  there  exists  an 
M  —  2P  dimensional  orthogonal  subspace,  denoted  V2,  which  is  spanned  by  the  column 
vectors  in  the  matrix  A, 

A  =  [ai,  02, . . .  Oa/_2p]  (3.103) 

For  any  vector  ip  eVi  and  7  €  V2  then  ip^y  =  0  (83:132-152).  In  particular,  Afoj  =  0  for 
t  =  1 . . .  2P  and  j  =  1 . . .  M  -  2P.  Now  let  F  be  an  M  by  M  invertible  matrix  and  define 
the  linear  transformation  upon  the  two  sets  of  basis  vectors  as  ip^  =  FXi,  where  V’j  €  V\, 
and  'fj  =  F~^aj,  where  7^  €  V2.  In  matrix  form, 

<if  =  [iP,...  iP2p]  =  F[Ai . . .  A2p]  =  FA  (3.104) 

r  =  [7i  •  •  ■  7^-2^]  =  •  •  •  aM-2p]  =  P-^A  (3.105) 

The  columns  of  'i'  form  a  basis  for  V\  while  the  coliunns  of  F  form  a  basis  for  V2  (26:92). 
The  projection  of  any  vector  z  E  into  Vj  and  V2  can  be  written  in  terms  of  the  matrices 
and  r  as  (83:156),(73:46-47) 

zi  =  for  zi  €  Vj  (3.106) 

Z2  =  r  [r^r] r’’z;  for  Z2  e  1^2  (3.107) 
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The  signal  energy,  E,  defined  by  the  vector  two  norm  l|z|p  for  z  G  ,  is  projected 
orthogonally  into  each  subspace  (26:181-226)  as  E  =  z  =  Ei  +  E-i  where 

El  =  zlzi  =  [/  -  r(r^r)-ir^]  z  (3.108) 

Ej  =  z[z2  =  z^r  [r^r]"‘  r^z  =  z^[i-  z  (3.109) 

For  a  fixed  energy  E,  minimizing  Ei  maximizes  Ej  and  vice  versa.  Now  let  E  be  defined 
by  the  Cholesky  decomposition  of  the  noise  covariance  matrix  so  that 

=  F'^F  (3.110) 

The  ML  firequency  objective  function,  J{f),  can  then  be  written  as 

J(/)  =  y'^F'^Fy  -  y^E’’EA(A’’E^EA)-iA^E^Ey  (3.111) 

For  z  =  Fy,  and  =  FA,  an  alternate  form  of  J(/)  becomes 

J(/)  =  Z^Z  -  (3.112) 

Factoring  the  vector  z  and  employing  Equation  3.109  yields 

j(/)  =  z^  [/  -  ^  ^  z^r(r^r)-'r^z  (3.113) 

Minimizing  the  ML  frequency  objective  function,  J(/),  is  equivalent  to  minimizing  E2, 
the  energy  of  the  signal  projected  into  Vj.  From  Equation  3.109,  since  F  =  F~^A,  this 
minimization  can  be  written  in  terms  of  the  basis  vectors  Oj  in  A  as 


J(A)  =  y^F^F-'^A{A^F-^F-^A)-^A^F-^Fy  =  y'^  A{A^  K^A)-^  A^y  (3.114) 

Minimizing  the  ML  frequency  objective  function  defined  by  Equation  3.25  with  respect 
to  /  has  been  recast  eis  minimizing  Equation  3.114  with  respect  to  the  basis  vectors  in 
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the  matrix  A.  Proper  selection  of  these  basis  vectors  will  prove  critical  for  obtaining  a 
simplified  estimator. 

3.6.2  Determination  of  the  Basis  Vectors.  To  determine  an  optimal  set  of  basis 
vectors,  a*,  consider  the  LP  model  of  Equation  2.14.  Substitution  of  s[Tn]  =  6*  cos{u;i.'m  +  <t>k) 
and  expanding  yields 

P  2P 

EE  a[p]bk{cos{(jJk[m-p])cos{(f>k)-sm{uJk[Tn-p])sm{4>k)}  =  0  (3.115) 

*=1 p=0 

Since  Equation  3.115  must  hold  for  all  u;*  and  2P  <  m  <  M  —  I,  using  the  substitution, 
=  6jtsin(<^t)  and  bl  =  6*cos((^n,),  gives  the  property 

2P 


a[p]  cos(a;t[m  —  p])  =  0  (3.116) 

p=0 

2P 

y^a[p]sin(u;t(m -p])  =  0  (3.117) 

p=0 

Now  define  the  vector  of  k  zeros  as  0*  and  the  M-dimensional  vector  a*  as 

0j^_2P-fc-i]  (3.118) 

for  fc  =  0 . . .  M  —  2P  —  1.  Since  the  form  of  the  beisis  vectors  for  Vi  can  be  written  as 

A<  =  [cos(27r/i[Afo  -  1]) . . .  cos(27r/i[iV  -  1])]^  (3.119) 

Ap+i  =  -[sin(27r/j[Af<,  -  1]) . . .  sm{2z fi[N  -  1])}^  (3.120) 


for  i  =  1 . . .  P,  then  a*  A^  =  0  for  j  =  1 . . .  2P.  The  M  —  2P  vectors  a*  are  orthogonal  to 
the  columns  of  A.  Furthermore,  employing  these  vectors  as  the  columns  of  A  yields  the 
LP  coefficient  matrix  defined  by  Equation  3.44.  The  M  -  2P  columns  of  A  are  linearly 
independent  and  thus  form  a  basis  for  Vj,  the  subspace  orthogonal  to  Vi.  In  addition,  each 
column  of  A  is  derived  from  a  single  vector  o  so  that  J{A)  can  be  written  as 

J(a)  =  y^A{A^KwA)~^A^y  (3.121) 
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Minimization  of  J{f)  with  respect  to  /  has  been  recast  as  a  minimization  of  J{a)  with 
respect  to  a.  The  LP  coefficients  which  minimize  Equation  3.121  provide  ML  frequency 
estimates  (6).  Furthermore,  via  the  invariance  property  of  ML  estimators,  the  a  which 
minimizes  J{a)  is  the  ML  estimate  of  the  LP  coefficients(40:185).  As  shown  in  the  next 
section,  this  ML  estimate  of  the  LP  coefficients  can  also  be  obtained  from  the  LP  general 
linear  model. 

3. 6. 3  Relationship  to  LP  Model.  To  show  the  relationship  between  ML  frequency 
estimation  and  the  LP  general  linear  model,  let  G{a)  be  the  square  M  —  2P  matrix  obtained 
from  the  Cholesky  decomposition  of  the  matrix  ■  The  ML  frequency  objective 

function  can  now  be  written  as 

J(a)  =  y'^AG'^{a)G{a)A^y  (3.122) 

Employing  the  M  -  2P  by  2P  + 1  matrix  of  measurements,  Yp,  and  imposing  the  sinusoidal 
constraints,  so  that  a  =  Ba,  yields  A^y  =  Ypa  —  YcOt.  Thus,  an  edteniative  form  of 
Equation  3.121  becomes 

J(a)  =  a'^YjG{afG{a)Yca  (3.123) 

By  defining  the  error  vector  as  e{a)  =  [e[M  —  1] . . .  e[2P]]^  and  constraining  q[0]  =  1 
yields 

e(a)  =  G{a)Yca  -  y^ict)  +  Tc(q)Qo  (3.124) 

This  equation  is  identical  to  the  LP  general  linear  model  given  by  Equation  3.53.  By  proi> 
erly  incorporating  the  effects  of  the  filter  coefficients  and  the  LP  coefficients  in  developing 
the  LP  general  linesu"  model,  this  derivation  shows  the  exact  relationship  between  the  LP 
coefficients  and  ML  frequency  estimates.  The  set  of  LP  coefficients  which  minimize  the 
squared  error  defined  by  the  LP  general  linear  model  will  provide  ML  estimates  of  the 
frequencies.  Furthermore,  via  the  invariance  of  ML  estimators,  the  LP  coefficients  which 
minimize  the  squared  error  are  the  ML  estimates  of  the  coefficients. 
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3.6-4  Section  Summary.  This  section  derived  the  exact  relationship  between  the 
ML  frequency  objective  function  for  filtered  sinusoids  and  the  LP  general  linear  model 
and  is  an  original  contribution  of  this  research.  Specifically,  this  section  showed  the  set 
of  LP  coeflficients  which  minimize  the  square  error  defined  by  the  LP  general  linear  model 
will  provide  ML  frequency  estimates  and  vice  versa.  The  next  section  will  derive  an  ML 
estimator  for  the  LP  coefficients  and  derive  the  relationship  between  this  estimator  and 
the  ILS  and  ITLS  fixed  point  estimators. 

3.7  LP  Coefficient  Estimation:  Objective  Function  Minimization 

In  this  section,  after  deriving  a  method,  termed  Iterative  Exact  Gradient  Descent 
(lEGD),  for  obtaining  exact  ML  frequency  estimates  using  gradient  descent  algorithms 
to  minimize  the  constrained  form  of  the  LP  objective  function,  J{a),  approximate  ML 
estimation  techniques  are  examined.  Specifically,  this  section  will  show  the  LP  coefficients 
found  via  the  ILS  fixed  point  mapping  function  are  exactly  equivalent  to  the  Iterative 
Generalized  Least  Squares  (IGLS)  approximate  ML  estimator  developed  in  (31),(32)(70). 
In  addition,  the  LP  coefficients  found  via  the  ITLS  fixed  point  mapping  function  will  be 
shown  to  be  exactly  equivalent  to  the  well  known  Iterative  Quadratic  Maximum  Likelihood 
(IQML)  approximate  ML  estimator  proposed  in  (6),  (29),  (30),  (68),  (63), (75).  The  proof 
of  the  equivalence  of  these  estimators  is  an  original  contribution  of  this  dissertation  and 
correctly  casts  the  IGLS  and  IQML  estimators  as  fixed  point  mappings. 

3.7.1  Iterative  Exact  Gradient  Descent  (lEGD).  From  the  form  of  the  objective 
fxmction  as  defined  in  Equation  3.123,  the  interpretation  as  the  the  familiau  squared 

error  objective  function  is  self  evident 

J(q)  =  e{afe{a)  =  a'^Yj  G(afG(a)Yca  (3.125) 

Minimizing  J(a)  wdth  respect  to  a  becomes  a  F  +  1  dimensional  search  for  an  estimate, 
d,  such  that 

d  =  ar5{imn(a^y'J’G(a)^G(Q)yc'Q)}  (3.126) 
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The  Q  which  minimizes  J{ot)  will  provide  ML  estimates  of  the  frequencies  and  the  LP 
coefficients. 

There  are  many  methods  for  minimizing  J{oi)  with  respect  to  a.  The  method  devel¬ 
oped  in  this  dissertation,  termed  the  Iterative  Exact  Gradient  Descent  (lEGD)  algorithm,  is 
based  on  an  iterative  gradient  descent  algorithm  of  the  general  form  (15:140-141),  (40:187), 


=  a‘  - 


D 


_,dJia) 


da 


a=a’ 


(3.127) 


where  the  vector  dJ{a)/da  is  the  P  -I- 1  dimensional  gradient  vector  defined  as  (40:187) 


•3J(a)1  dJ{a) 
da  .  j  9a  [t] 


In  addition,  D  is  the  Hessian  of  J{a)  with  (40:187) 


[Dkj 


a^Jja) 

9a[i]9a[j] 


(3.128) 


(3.129) 


As  shown  in  Appendix  E,  the  gradient  is  relatively  simple  to  implement  while  the  Hessian 
is  a  bit  more  complicated.  Again,  with  most  iterative  optimization  techniques,  there 
is  no  guarantee  the  algorithm  will  ever  converge.  Even  if  it  does  converge,  there  is  no 
guarantee  the  global  minimum  hsis  been  attmned.  The  solution  achieved  will  only  provide 
a  local  minimum  and  therefore  the  accuracy  of  the  estimate  will  depend  greatly  on  the 
location  of  the  initizd  estimate. 

Figmre  3.13  shows  typical  solutions  for  the  LP  coefficients  obtained  using  the  lEGD 
algorithm.  In  this  figure,  the  results  of  the  ILS  fixed  point  estimator  were  used  as  an 
initial  guess  for  the  lEGD  algorithm.  This  figmre  shows  the  lEGD  algorithm  does  indeed 
find  the  minimum  of  J{a)  and,  consequently,  the  ML  estimate  of  the  LP  coefficients  and 
frequencies.  The  main  drawback  to  the  lEGD  algorithm,  when  compared  to  the  ILS  and 
ITLS  fixed  point  algorithms,  is  the  added  complexity  of  computing  the  Hessian  at  each 
iteration.  Typical  methods  for  reducing  this  complexity  treat  the  matrix  AJKwA  as  a 
constant,  impose  a  constraint  on  a  and  perform  the  minimization  with  respect  to  this 
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constraint  (6), (9),  (29),  (31), (32), (75).  The  two  estimators  based  on  this  technique  have 
been  termed  the  IGLS  and  IQML  estimators. 

3.1.2  Iterative  Generalized  Least  Squares  (IGLS).  One  method  of  performing  an 
approximate  minimization  is  to  constrain  a[0]  =  1  and  perform  a  gradient  descent  using 
Equation  3.127  with  the  matrix  product,  C(Qo)  =  A^K^A,  treated  as  a  constant.  For 
this  constraint,  the  objective  function  becomes  a  function  of  the  P  remaining  terms  in  ag 

=  yJlCiair^y,  +  2yJ[C{ai)\-^Ygag  +  a^F/[C(a‘)]-^y,a,  (3.130) 


where  ^(aj,)  is  derived  from  the  previous  estimate  of  Qg.  Taking  the  gradient  and  evalu¬ 
ating  at  Q*  yields  the  general  iterative  gradient  descent  algorithm 

=  a‘  -  2Z)-i  {y7[C(a‘)]-‘na‘  +  ^/[^(q* )]-^y  J  (3.131) 


Employing  the  Hessian  matrix,  D,  yields 


D  =  2y/[C(a‘)l-‘n 


(3.132) 


Thus,  the  final  form  of  this  iterative  gradient  descent  algorithm  becomes 

=  -  {r/[c(Q‘)]-‘y,}-'  y/[c(Qt)]-^i/,  (3.133) 

Since  [C(a’)]“^  =  G^(Q’)G(a’),  and  yc(o[o)  =  G(a‘)yc,  substitution  yields 

=  -  [Yg^ i<)Yg{ai)] YJ (ai)vM)  (3.134) 

The  approximate  minimization  of  the  ML  objective  function  is  identical  to  the  ILS  fixed 
point  mapping  function,  £/£5(a),  defined  in  Equation  3.84.  This  derivation  is  an  orig¬ 
inal  contribution  of  this  dissertation  for  it  establishes  the  connection  between  the  IGLS 
algorithm  developed  in  (31),  (32)  and  (70)  to  the  general  method  of  gradient  descent 
minimization  and  correctly  casts  the  IGLS  algorithm  m  a  fixed  point  ILS  mapping. 
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Convergenca  of  lEDQ  Aigorlthm 


(a)  Sinusoid  Parameters:  [6°  =  l,/i  =  .192,  d>i  =  fr/3],  and  SNR  =  -5dB. 
Objective  Function  Evaluation;  J{oiiegd)  =  35.009,  Jmin  =  35.009. 


Convergence  of  lEDQ  Algorithm 


(b):  Sinusoid  Parameters:  [6°  =  l,/i  =  .207, <^i  =  tt/S],  [ftj  =  l,/2  =  -227,02  =  47r/3]. 
Objective  Function  Evaluation:  J{ajEGD)  =  3.5230,  Jmin  =  3.5230.,  SNR  =  5dB. 

Figtire  3.13  lEGD  Algorithm  Convergence;  J(a)  calculated  as  function  of  LP  co¬ 
efficients  at  fixed  SNR  and  block  length  M  =  32.  Ten  iterations  allowed  for 
convergence.  Filter  Parameters:  Center  Frequency:  fc  =  -21;  Bandwidth;  /jg  =  .2; 
Length:  N  =  32.  SNR  calculated  as  SNR  =  -101ogio(2«r^). 
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3.7.3  Iterative  Quadratic  Maximum  Likelihood  (IQML).  An  alternative  method 
of  performing  the  approximate  minimization  of  J(a),  with  C(a)  =  A^Kw^  held  constant, 
is  to  constrain  a  to  be  of  unit  length,  a^a  =  1,  and  iteratively  minimize  J{a)  via  an 
eigenvalue  analysis.  That  is,  rewriting  J(a)  as 

J(a)  =  a^YjlC(a*)j-^Vca  =  a^Vj(a')Vc(a‘)a  (3.135) 

Again,  C(a')  represents  a  constant  matrix  obtained  from  the  previous  estimate  of  Qj. 
Using  the  method  of  Lagrange  Multipliers  yields  the  objective  fimction  (49:897-923) 

Ji(a)=  a^Vj (a')Vc(a*)a  -  Aa^a  (3.136) 

Taking  the  gradient  with  respect  to  a  and  equating  to  zero  produces 

2Yj (a*)Yc(a')a  -  2Aa  =  0  (3.137) 

Under  this  unit  length  constraint,  a  is  an  eigenvector  of  the  matrix  product  {Yq  (a’)Fc(o:')} 
Substituting  into  J{a.)  gives 

J(q)  =  Aa^Q  =  A  (3.138) 

The  optimal  estimate  of  a  is  the  eigenvector  associated  with  the  smallest  eigenvalue  of  the 
matrix  product  {y'(J(a*)yc(c»')}-  Thus,  for  the  IQML  algorithm,  each  time  an  estimate  of 
a  is  obtained,  it  is  reinserted  into  J(a)  to  produce  a  new  estimate.  This  process  continues 
until  convergence  of  the  algorithm.  An  alternative  explanation  of  this  algorithm  can  be 
obtained  by  considering  the  SVD  of  Yc(a*)  directly.  That  is,  let 

yc(a)  =  Cl(Q)E(a)V^(a)  (3.139) 

Here,  C/(q)  and  V(a)  are  unitaury  matrices.  Writing  in  terms  of  column  vectors  yields 

U(a)  =  [ui(a) . . .  tiM-2i’(a)]  with  u*  e  (3.140) 

U(q)  =  [vi(q)  . . .  t;p+i(a)]  with  e  (3.141) 
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In  addition,  E(a)  is  an  Af  -  2P  by  P  +  1  diagonal  matrix  of  the  singular  values  of 
denoted  cTj  for  i  =  1 . . .  P  +  1,  and  arranged  so  that  (Ti  >  cr^  ■  ■  ■  >  crp+i  >  0.  Thus,  in  terms 
of  the  unitary  matrix  V{ai) 

Yj{a')Yc{a')  =  y(Q')E^(Q‘)E(a‘)V^(a‘)  =  y(a‘)A(a‘)y^(a‘)  (3.142) 

Here,  A(q‘)  is  the  diagonal  matrix  of  the  eigenvalues  associated  with  Yq  {q.')Yc{o.').  Since 
Aj  =  (7?,  the  eigenvector  associated  with  the  smallest  eigenvalue  is  Vp+i(a‘),  the  right-most 
singulzu:  vector  obtained  directly  from  the  SVD  of  Vc(c«‘)-  That  is 

=  tjp+i(a‘)  (3.143) 

This  solution  is  exactly  the  same  as  the  unnormalized  fixed  point  solution  defined  by  the 
ITLS  fixed  point  mapping,  Equation  3.100.  This  derivation  is  an  original 

contribution  of  this  dissertation  for  it  equates  the  IQML  algorithm,  previously  developed  as 
a  minimization  algorithm,  to  an  ITLS  fixed  point  solution  for  a.  What  has  been  interpreted 
in  the  past  as  an  approximate  minimization  is  actually  a  fixed  point  determination  for  the 
LP  coeflBcients. 

S.7.^  Algorithm  Comparison.  Figure  3.14  shows  the  average  difference  in  the 
squared  error,  at  convergence,  between  the  lEGD  and  ILS  algorithms,  and  the  lEGD  and 
ITLS  algorithms,  for  the  sinusoids  tested.  This  figure  shows  the  ILS  estimator  produces, 
on  average,  a  smaller  squared  error  of  the  LP  objective  function  then  that  produced  by 
the  ITLS  algorithm.  Hence,  the  ILS  estimator  will  provide  LP  estimates  closer  to  the  ML 
estimates  then  those  produced  by  the  ITLS  estimator.  Figure  3.15  shows  the  fi:equency 
estimation  accuracy  of  each  of  these  algorithms  for  both  one  and  two  filtered  sinusoids. 
As  expected,  for  this  data,  the  lEGD  algorithm  provides  the  most  accurate  frequency 
estimates.  However,  these  estimates  are  only  shghtly  better  than  those  provided  by  the 
ILS  and  ITLS  fixed  point  estimates,  especially  at  large  SNRs.  This  slight  improvement 
must  be  weighed  against  the  additional  complexity  of  calculating  the  Hessian  matrix  for 
the  lEGD  algorithm  at  each  iteration. 
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Difference  in  MSE 


Difference  in  MSE 


(b)  Sinusoid  Paurameters:  [6°  =  l,/i  =  .207, <^1  =  5r/3],  [62  =  I1/2  =  ■227,<j>2  =  47r/3]. 


Figure  3.14  LP  Estimator  Minimum  Error  Comparison:  J(a)  evaluated  at  con¬ 
vergence  for  each  algorithm.  MSE  calculated  at  SNR  intervals  of  .5dB  from  500 
independent  realizations  of  M  =  32  samples  of  sinusoids  in  noise.  Ten  iterations 
allowed  for  convergence  of  EEGD,  ELS  and  ITLS  algorithms.  Minimum  error  found 
from  EEGD  algorithm  initialized  via  a  at  ELS  convergence.  Filter  Parameters: 
Center  FVequency:  fc  =  .21;  Bandwidth:  fs  =  .2;  Length:  N  =  32.  SNR  calculated 
as  SNR  =  -101ogio(2<T2).  MSE  =  3^  [JiaiEDa]  - 
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-10log10(MSE)  -10log10(MSE) 


Frequency  Measurement  Accuracy 


(a)  Sinusoid  Parameters:  [6°  =  l,/i  =  .192,  =  ir/S]. 


Combined  Freduoncy  Measurement  Accuracy 


Figiire  3.15  LP  Estimator  Frequency  Estimation  Compeu-ison:  MSE  calculated 
at  SNR  intervals  of  .5dB  from  500  independent  realizations  of  M  =  32  sam¬ 
ples  of  sinusoids  in  noise.  Ten  iterations  allowed  for  convergence  of  each  algo¬ 
rithm.  lEGD  dgorithm  initialized  via  a  at  ELS  convergence.  Filter  Parameters: 
Center  Frequency:  fc  =  .21;  Bandwidth:  fs  =  -2  Length:  N  =  32.  SNR  calculated 

as  SNR  =  -101ogio(2a2)  and  MSE  =  ^  Ep=i  [/p(0  "  fpf- 
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5.7.5  Section  Summary.  This  section  developed  an  exact  ML  estimator,  termed 
the  lEGD  algorithm,  for  the  LP  coefficients  baised  on  the  LP  general  linear  model  and  is 
an  original  contribution  of  this  research.  The  estimator  derived  provides  ML  estimates  of 
the  LP  coefficients  and,  consequently,  the  frequencies  of  filtered  sinusoids,  by  employing 
an  iterative  gradient  descent  algorithm  to  minimfre  the  squared  error  dictated  by  the  LP 
general  linear  model.  In  addition,  this  section  derived  the  equivalence  between  the  IGLS 
and  IQML  algorithms  and  the  ILS  and  ITLS  fixed  point  estimators  respectively.  These 
derivations  show  the  IGLS  and  IQML  algorithms  are  fixed  point  estimators  and  do  not 
minimize  the  LP  objective  function  in  general.  Simulations  show  the  lEGD  algorithm 
can  provide  ML  frequency  estimates  by  minimizing  the  LP  objective  function,  though  the 
performance  increase  over  the  less  complicated  ILS/IGLS  and  ITLS/IQML  algorithms  is 
negligible  at  high  signal  to  noise  ratios.  In  addition,  for  the  simulations  completed  the 
ILS/IGLS  algorithm  actually  produced,  on  average,  more  accurate  frequency  estimates 
than  the  more  complicated  ITLS/IQML  algorithm.  The  IQML  algorithm  requires  an 
eigenvalue  decomposition  ofaP  +  lbyP+1  squzure  matrix  at  each  iteration  wherezis 
the  ILS  algorithm  only  requires  the  inversion  of  a  P  by  P  squEire  matrix.  This  result  is 
surprising  since  the  IQML  algorithm  has  been  widely  accepted  eis  the  premier  method  of 
estimating  the  frequencies  of  complex  sinusoids  in  noise.  Thus,  if  the  small  degradation  in 
performance  over  the  lEGD  can  be  tolerated,  the  ILS/IGLS  estimator  should  be  preferred 
over  the  TLS /IQML  estimator  for  providing  minimum  variance  unbieised  point  estimates  of 
the  LP  coefficients  and,  consequently,  the  sinusoidal  frequencies.  The  next  section  develops 
a  new  method  for  determining  the  estimation  accuracy  for  each  point  estimate  of  the  LP 
coefficients  and  frequencies  based  solely  on  a  single  realization  of  the  meEisurements,  y. 

3.8  Bounding  the  Estimation  Error 

Previous  portions  of  this  dissertation  have  developed  methods  of  obtaining  unbiased 
point  estimates  of  the  LP  coefficients  and,  consequently,  the  sinusoidal  frequencies  present 
in  a  set  of  M  observations.  However,  knowledge  that  an  estimate  is  unbiased  is  not  ade¬ 
quate  to  fully  characterize  the  estimates;  the  estimation  error  must  also  be  quantified.  For 
sinusoids  in  noise,  the  accuracy  of  each  estimate  is  a  function  of  several  parameters  includ- 
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ing  the  SNR,  the  number  of  measurements  and  the  relative  amplitudes,  frequencies  and 
phases  of  the  sinusoids  (39:414,415).  This  section  introduces  a  new  method  for  bounding 
the  estimation  error  of  the  LP  coefficient  estimates  and  consequently,  the  frequency  esti¬ 
mates,  based  strictly  on  one  realization  of  the  measurement  vector.  Hence,  the  accuracy 
estimate  is  data  driven.  This  derivation  is  an  original  contribution  of  this  research  and 
can  be  used  to  establish  confidence  intervals  for  both  the  LP  coefficients  and  the  frequency 
estimates.  In  awldition,  knowledge  of  the  accuracy  of  the  estimate  is  critical  for  data  fusion 
algorithms  and  pattern  classification  systems. 

3.8.1  Theory.  Let  6  represent  the  true  values  of  a  set  of  P  parameters  and  let  B 
be  an  associated  estimate.  The  estimation  error  of  any  one  of  the  parameters,  denoted  Cp 
will  be  defined  as  Op  (61:345). 

€p  =  Op-  Op  (3.144) 

As  a  function  of  a  random  variable,  this  error  is  also  a  random  variable  and  can  best  be 
analyzed  in  probabilistic  terms.  That  is,  if  t  is  an  arbitrary  constant,  the  probability  the 
absolute  error  is  less  then  t  is  found  as 

P(|cp|  <  t)  =  P{-t  <€p<t)  (3.145) 

Calculation  of  this  probability  depends  on  the  PDF  of  Cp  which  is  usually  unknown.  Pro¬ 
vided  the  estimate  is  unbiased,  however,  a  loose  bound  on  this  probability  of  error  can 
be  found  using  only  the  variance  of  the  estimate  by  employing  Chebyshev’s  Inequality 
(27:58-59) 

P(kpl  >  0  <  ^  (3.146) 

Letting  t  be  an  integer  multiple  of  the  standard  deviation,  t  =  kcTg^ ,  yields 

P(kpl  >  k<Tg^)  <  p  (3.147) 
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Thus,  if  the  variance  of  an  estimate  for  dp  can  be  calculated,  a  loose  bound  on  the  estimation 
error  can  be  achieved.  Alternatively,  confidence  intervals  can  be  formed  ^ls 


P{dp  -  <6p  <  0p  +  kcTgJ  =  1-/3  (3.148) 

Here  the  quantity  (1  —  /9)  is  called  the  confidence  coefficient  and  indicates  the  relative 
frequency  the  interval  defined  by  the  estimate  and  the  variance  of  the  estimate  will  include 
the  true  value  of  9p  (28:151-158).  Again,  this  calculation  depends  on  the  exact  form  of  the 
PDF  of  the  error,  although  a  bound  can  be  obtained  from  Equation  3.147. 

Thus,  if  the  variance  of  an  unbiased  estimate  is  known  or  can  be  calculated  from 
a  single  measurement  vector,  the  accurzu:y  of  the  estimate  can  be  gauged  by  employing 
Equation  3.147  or  a  confidence  interval  can  be  established  via  Equation  3.148.  A  method 
to  estimate  the  variance  of  a  point  estimate  of  the  LP  coefficients,  using  only  the  measure¬ 
ments  and  a  knowledge  of  the  variance  of  the  noise,  is  derived  in  the  next  section. 

3.8.2  LP  Coefficients.  To  estimate  the  variance  of  an  estimate  of  the  LP  coeffi¬ 
cients,  consider  the  effect  of  the  system  model  on  the  measurements.  From  the  LP  general 
linear  model  of  Equation  3.49,  since  j/[m]  =  s[m]  -i-  iy[m]  then 

G{a)Ya  =  e{a)  =  G{a)Sa  +  G{a)A^w  (3.149) 

With  a[0]  =  1,  this  form  of  the  LP  general  linear  model  can  be  written  as 

[e(a)  -  G(a)So]  =  G{a)Soao  +  G{a)A^w  (3.150) 

where  Sg  is  the  first  column  of  5  and  5<,  is  an  M  —  2P  by  P  matrix  of  the  remaining 
columns.  The  ML  estimate  for  o<,  is  unique  and  found  eis  (40:186), (60) 

ao{ML)  =  [5jG^(a)G(a)5,]"‘  {5jG^(a)[e(a)  -  G(a)s„]}  (3.151) 
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In  addition,  the  statistical  properties  of  this  estimate  are  found  2is  (40:186) 

E{do{ML)}  =ao  (3.152) 

Kd.  =  [5jG^(a)G(a)S,]“'  •  (3.153) 

This  gives  the  covariance  matrix  for  the  ML  estimate  of  the  LP  coeflBcients  in  terms  of 
the  deterministic  model  matrix  Sg.  Unfortunately,  the  elements  of  this  matrix  are  derived 
from  the  deterministic  signal,  s[Tn],  which  is  unavailable.  To  estimate  the  covariance  matrix 
from  the  observations,  consider  the  expansion  of  the  observation  matrix  defined  by  G{a)Yo. 
Letting  L  =  M  —  2P,  then 

9ui  9u2  •  •  •  9i,L  y[M  -  2]  y[M  -  3]  ...  y[L  -  l] 

92.1  92,2  •••  92,l  y[M-3]  y[M  -  4]  ...  y[L  -  2] 

91.1  9l,2  ■  ■  •  9l,l  .  .  y[2P  -  1]  y[2P  -  2]  ...  y[0] 

Thus,  each  element  of  this  matrix  product  can  be  viewed  as  the  output  of  a  finite  impulse 
response  filter  of  length  L.  That  is,  for  i  =  1 ...  L  and  j  =  1 . . .  2P, 

t-i 

[G{a)Yo\^j  =  9i,m+iylM  -m-l-j]  (3.155) 

m=0 

Now  define  the  vector  yj  =  {y[M  —  2], ... ,  y[0]]^  and  the  matrix 
filter  matrix 

9i,i  ■  ■  ■  9i.L  0  0  ...  0 

0  9i,i  •  •  •  9i.L  0  ...  0 

:  :  :  :  0  ;  0 

0  0  ...  0  yi,i  ... 

Then,  the  row  of  G{a)Yo  can  be  written  as  yjGi.  Thus 

YjG'^{a)=[Gjy,...Gly,]  (3.157) 


j  as  the  M  -  1  by  2P 


(3.156) 


G{a)Yg  = 


(3.154) 
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As  a  result,  the  matrix  product  [yj^G^(a)G(a)y'(,]  can  be  written  as 

y/G^(a)G(a)y,  =  fi  ^ly^vlGm  (3.158) 

m=l 

Taking  the  expected  value  of  this  matrix  yields 

£;{y/G’’(a)G(a)y,}  =  ^  GlE{y,yJ}G,n  (3.159) 

m=l 

Now,  with  and  Wi  defined  in  a  manner  similar  to  y^,  then  y^  =  8i  +  Wi,  so  that 

Ely^yJ}  =  3i3j  +  E{wiw'^}  (3.160) 

Substitution  into  Equation  3.159  produces 

E{y/G^(a)G(a)y,}  =  Gls.s.G^  +  ^  }G„  (3.161) 

m=l  m=l 

Since  Wi  is  a  zero-mean,  normally  distributed  random  vector  with  covariance  matrix 
Kw  =  cr^H'^H,  then 

E;{y/G^(a)G(a)y,}  =  5jG^(a)G(a)5„  +  G^H^HG^  (3.162) 

m=l 

For  any  ML  estimate  of  a,  an  estimate  of  the  covariance  matrix  can  be  obtained  as 

[■  M-2P 

kc^=  y/G’’(a)G(a)n-(72  ^  GIH'^HG,,  (3.163) 

m=l 

Finally,  employing  the  symmetry  constraints,  a  =  Ba.  with  a[0]  =  1,  the  final  form  of  the 
estimate  for  the  covariance  matrix  of  the  constrained  coeflBcients  in  Qq  becomes 

r  M-2P  •]~i 

=  y7G^(a<,)G(a,)y,-a2S^  GIH^HG„,B  (3.164) 

L  m=l 
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where  Yc  =  YoB.  For  large  signal  to  noise  ratios,  assuming  Qo  is  relatively  close  to  the 
true  value  of  Qo,  an  estimate  of  the  LP  coefficient  covariance  matrix.  becomes 

(3.165) 


Thus,  using  only  the  measurements  and  exploiting  a  knowledge  of  the  noise  variance,  the 
variance,  of  each  LP  coefficient  can  be  estimated  as 


V{a[p]}  =  [k^ 


(3.166) 


Substitution  of  this  estimate  into  Equation  3.147  yields  a  bound  on  the  estimation  er¬ 
ror.  Alternatively,  substitution  of  this  estimate  into  Equation  3.148  provides  a  confidence 
interval. 

Figvire  3.16  shows  the  accuracy  of  the  variance  estimates  of  the  LP  coefficients  ob¬ 
tained  from  the  ILS  estimator  for  one  and  two  filtered  sinusoids.  For  both  cases,  the  average 
value  of  the  variance  estimates  coincides  with  the  true  variance  of  the  LP  estimates  as  given 
by  the  CRLB.  In  addition,  the  variance  of  these  variance  estimates  is  extremely  small.  This 
indicates  each  estimate  of  the  vzuriance  will  be  relatively  close  to  the  true  value  of  the  vari¬ 
ance  so  that  Equation  3.147  and  Equation  3.148  can  be  used  to  aiccurately  characterize 
the  accuracy  of  a  point  estimate  of  the  LP  coefficients.  This  information  can  also  be  used 
to  determine  the  accuracy  of  the  ^lssociated  firequency  estimates. 


3.8.3  Sinusoidal  Frequencies.  Previously,  a  method  was  derived  to  estimate  the 
variance  of  the  LP  coefficients  firom  a  single  measurement  vector  and  knowledge  of  the 
noise  variance.  To  map  this  estimate  to  the  frequency  space,  a  transformation  must  be 
employed.  In  general,  let  d  be  an  unbiased  estimate  of  a  and  let  be  the  covariance 
matrix  of  the  estimate.  In  addition,  let  fp  =  gp{a)  be  a  scalar  function  of  a.  The  variance 
of  fp  can  be  estimated  as  (19:183),(20). 


v{U} « 


dgpia) 


■,T 


da 


k 


a 


'dgpjay 

da 


a=a 


(3.167) 
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Thus,  for  a  scalar  function,  gp{a),  an  estimate  of  the  variance  can  be  obtained  provided 
the  first  derivatives  exist  and  a  is  not  too  far  from  a. 

In  general,  there  is  no  closed  form  function  which  relates  the  coefficients  {a[p]}  di¬ 
rectly  to  the  frequencies  {/p}.  However,  for  P  =  1  and  P  =  2,  and  to  a  certain  extent  for 
P  =  3  and  P  =  4,  closed  form  functions  relating  the  frequencies  to  the  coefficients  can  be 
derived  which  allow  Equation  3.167  to  be  employed  to  bound  the  estimation  error  for  the 
frequencies. 

For  one  sinusoid,  from  Equation  3.57,  From  Equations  the  function  relating  /i  to  the 
coefficient  a[l]  is  given  as 

=  5i(a(l])  (3.168) 


/,  =  —  cos  ^ 
2tt 


On  the  interval  0  <  /i  <  .5,  then  -2  <  a[l]  <  2  and  the  variance  of  the  frequency  estimate 
becomes  (48:462) 


V{h) 


47r2(4  —  a[l]2) 


V{a[l]} 


a[l]=a(ll 


(3.169) 


For  two  sinusoids,  employing  Equation  3.58  yields  the  relationships  (53) 


fp  —  5p(®^o) 


—  cos 
27r 


(up) 


(3.170) 


where  the  quantity  Up  is  defined  as 


Up 


-a[l]  -t-  (-l)'’+ya[l]2-4a[2]+8 


for  p  =  1, 2,  provided  a[ip  -  4a[2]  -b  8  >=  0.  Substitution  for  o[p]  yields 


(3.171) 


[cos(27r/i)  —  cos(27r/2)]^  >=  0 


(3.172) 


Since  this  inequality  holds  for  all  /,  the  mapping  between  /  and  a.o  exists.  Taking  the 
derivative  with  respect  to  each  parameter  yields 


dgp{ao) 


9«[1]  -  u 


-1  - 


(-l)Pa[l] 


d9p{oto) 


-  ul  L  -  4a[2]  +8] 


^a[l]2  -4o[2]  +  8] 
(-1)" 


Thus,  the  variance  of  the  frequency  estimates  becomes 


yUp]  = 


'gpiaoY 

.  dOo  . 


gp(Qo)' 
doio  . 


(3.173) 

(3.174) 


(3.175) 


These  variance  estimates  can  now  be  used  to  bound  the  estimation  error  or  develop  confi¬ 
dence  intervals  for  each  point  estimate  of  the  fi-equencies. 

Figure  3.17  shows  the  accuracy  of  the  frequency  confidence  intervals,  obtained  from 
Equation  3.148,  for  both  one  and  two  filtered  sinusoids.  Here,  the  ILS  algorithm  was  used 
to  determine  the  LP  coefficients  while  the  variance  estimates  for  the  LP  coefficients  were 
obtained  from  Equation  3.165.  The  variance  estimates  for  the  frequencies  were  calculated 
from  Equation  3.175.  For  this  data.  Figure  3.17  shows  the  percentage  of  time  the  confidence 
intervals,  developed  from  the  point  estimates,  enclosed  the  true  value  of  the  frequency  for 
one,  two  and  three  standard  deviations  of  the  variance  estimates.  In  each  ceise,  above  the 
SNR  threshold,  the  confidence  intervals  established  provide  an  accurate  method  of  gauging 
the  accuracy  of  the  point  estimate  of  the  frequency. 


3.8.4  Section  Summary.  This  section  derived  a  method  for  bounding  the  esti¬ 
mation  error  of  the  LP  coefficient  estimates  and  consequently,  the  frequency  estimates, 
based  strictly  on  one  realization  of  the  me^lsurement  vector,  and  is  an  original  contribution 
of  this  dissertation.  Specifically,  from  the  LP  general  linear  model,  this  section  derived  a 
method  for  estimating  the  variance  of  the  LP  coefficients  based  solely  upon  knowledge  of 
the  noise  varicince  and  the  set  of  measurements.  This  unbiased  estimate  of  the  LP  vciriance 
was  then  transformed  into  an  estimate  of  the  frequency  variance  for  one  and  two  sinusoids. 
Simulations  indicate  these  estimates  of  the  measurement  variance  can  be  used  to  gauge 
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%  Time  True  Both  Freqs  With  K-STDs  of  Estimate  %  Time  True  Freq  With  K-STDs  of  Estimate 


Frequency  Confidence  Intervals 


Frequency  Confidence  Intervals  for  Both  Frequencies 


Figure  3.17  Frequency  Confidence  Intervals:  Confidence  intervals  established  at  SNR 
intervals  of  .5dB  from  500  independent  realizations  of  Af  =  32  samples  of  sinu¬ 
soids  in  noise.  LP  coefficients  estimated  by  ILS  algorithm.  Ten  iterations  allowed 
convergence.  Filter  Parameters:  Center  Frequency:  fc  =  .21;  Bandwidth:  /s  =  .2; 
Length:  N  =  32.  Confidence  interval:  |/p(i)  -  fp\  <  kdf  for  k  =  1,2,3.  SNR  calcu¬ 
lated  as  SNR  =  -101ogig(2cr^). 
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the  accuracy  of  the  point  estimates  of  both  the  LP  coefficients  and  the  frequencies.  This 
gauge  can  then  be  employed  to  develop  confidence  intervals  to  aid  any  decision  making 
process  based  on  a  single  set  of  measurements. 

3.9  Summary 

This  chapter  derived  estimators  for  the  amplitude,  phase  and  frequency  parameters 
of  filtered  sinusoids  in  noise.  After  deriving  a  mathematical  model  describing  the  effects  of 
a  filter  on  sinusoids  and  noise,  ML  estimators  for  the  sinusoidal  parameters  were  developed 
b^lsed  on  this  model.  During  this  development,  it  was  shown  that  ML  estimates  of  the 
amplitude  and  phase  parameters  require  ML  estimates  of  the  frequencies  in  addition  to  the 
use  of  the  correct  system  model.  An  ML  estimator  for  the  firequencies  was  then  derived 
based  on  an  iterative  gradient  search  employing  the  ML  frequency  objective  function. 
Simulations  indicate  this  technique  can  produce  minimum  variance  unbiased  firequency 
estimates  provided  the  effects  of  the  filter  are  correctly  incorporated  into  the  system  model. 

Since  this  ML  frequency  estimator  can  require  many  iterations  to  achieve  conver¬ 
gence,  an  alternative  method  for  estimating  the  frequencies  was  developed  based  on  linear 
predictive  modeling.  Specifically,  an  equivalent  general  linear  model,  parameterized  by  a 
set  of  LP  coefficients  was  derived  for  estimating  the  frequencies  of  the  sinusoids  in  noise. 
The  LP  general  linear  model,  derived  in  this  chapter,  properly  accounts  for  the  effects  of 
the  LP  coefficients  and  system  filter  on  the  noise  and  is  an  original  contribution  of  this 
research.  Estimation  of  the  sinusoidal  frequencies  was  recast  as  the  estimation  of  the  LP 
coefficients  using  the  general  linear  model. 

The  ILS  and  ITLS  estimators  were  then  derived,  based  on  fixed-point  theory,  to 
estimate  the  LP  coefficients  of  the  general  linear  model  from  the  measurements.  The 
development  of  these  estimators  represents  an  original  contribution  of  this  research.  Simu¬ 
lations  indicate  each  of  these  estimators  produce  accurate  estimates  of  the  LP  coefficients 
and,  consequently  the  frequencies,  above  a  certain  SNR  threshold;  convergence  is  attained 
in  a  few  iterations. 
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This  chapter  then  related  the  ML  frequency  objective  function  for  filtered  sinusoids  to 
the  LP  general  linear  model.  The  analysis  showed  that  the  LP  coefficients  which  minimize 
the  sum  of  the  squared  errors  as  defined  by  the  LP  general  linear  model  also  provide  ML 
frequency  estimates  and  is  an  original  contribution  of  this  research.  An  ML  estimator  for 
the  LP  coefficients,  termed  the  lEGD  algorithm,  was  then  derived  based  on  an  iterative 
minimization  of  the  LP  general  linear  model  squared  error  and  is  an  original  contribution  of 
this  research.  Simulations  indicate  this  algorithm  will  provide  the  LP  coefficients  producing 
the  minimum  squared  error  and  consequently,  ML  estimates  of  the  LP  coefficients  and 
frequencies. 

Since  the  lEGD  algorithm  can  be  computationally  difficult  to  implement,  the  IGLS 
and  IQML  approximate  minimization  algorithms  were  examined.  Analysis  revealed  that 
the  IGLS  algorithm  is  equivalent  to  the  ILS  fixed  point  estimator  of  the  LP  coefficients 
while  the  IQML  algorithm  is  equivalent  to  the  ITLS  fixed  point  estimator.  This  recasting 
of  the  IGLS  and  IQML  algorithms  as  fixed  point  estimators  is  an  original  contribution  of 
this  research.  Simulations  comparing  the  performance  of  the  ILS  and  TLS  algorithms  with 
the  lEGD  algorithm  indicate  the  lEGD  algorithm  produces  frequency  estimates  which  are 
only  slightly  better  than  those  obtained  via  ILS  and  ITLS.  In  addition,  the  simulations 
indicated  the  ILS  algorithm  produces  frequency  estimates  at  least  as  accurate  as  those 
produced  by  the  ITLS  algorithm.  All  other  factors  being  equal,  the  ILS  algorithm  is  thus 
preferred  over  the  ITLS  algorithm  since  the  ITLS  algorithm  requires  an  iterative  SVD  of 
the  observation  matrix. 

Finally,  this  chapter  derived  a  method  of  bounding  the  estimation  error  of  the  point 
estimates  of  the  LP  coefficients  and  the  frequencies  based  only  on  the  measurements  and  a 
knowledge  of  the  noise  vari2ince.  This  derivation  employed  the  LP  general  lineeir  model  to 
obtain  unbiased  estimates  of  the  covariance  matrix  of  the  LP  coefficients  and  is  an  original 
contribution  of  the  research.  Simulations  indicate  the  estimates  of  the  variance  cire  good 
enough  to  allow  confidence  intervals  to  be  established  for  the  frequencies. 
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IV.  Application:  Digital  Electronic  Warfare  Receiver 
4.1  Introduction 

This  chapter  apphes  the  ILS  fixed  point  estimation  technique  to  the  digital  EW  re¬ 
ceiver  being  built  by  the  Air  Force  Research  Laboratory /Wright  Laboratory  (AFRL/WL). 
After  discussing  the  general  receiver  architecture,  models  of  the  real  and  complex  data  out¬ 
put  from  the  receiver  will  be  constructed.  A  method  to  efficiently  implement  the  receiver 
as  a  nonmaximally  decimated,  Uniform  Discrete  Fourier  Trcinsform  (UDFT)  polyphase 
filter  hank  is  then  derived.  After  showing  there  is  no  loss  in  frequency  estimation  accuracy 
attributable  to  the  use  of  the  complex  data  model  as  opposed  to  the  real  data  model,  a 
new  form  of  the  LP  general  linear  model  is  derived  for  complex  sinusoids.  Identification 
of  the  LP  coefficients  for  this  model  is  then  shown  to  be  related  to  ML  frequency  estima¬ 
tion  and  an  ILS  fixed  point  estimator  is  derived  for  estimating  the  LP  coefficients.  After 
applying  the  ILS  estimator  to  the  receiver  output  for  both  one  and  two  sinusoids,  the 
chapter  concludes  by  showing  the  PDF  of  the  complex  filtered  data  can  be  approximated 
as  a  complex  multivariate  Gaussian  PDF.  This  approximation,  in  turn,  allows  a  simplified 
frequency  estimator  to  be  derived  for  a  single  complex  sinusoid. 

4-2  Digital  Receiver  Architecture 

As  shown  in  Figure  4.1,  the  basic  structure  of  a  digital  channelized  receiver  can 
be  divided  into  the  antialiasing  filter,  Analog-to-Digital  Converter  (ADC),  filter  bank, 
demodulators,  decimators  and  parameter  encoders. 

In  general,  the  input  signal  can  be  modeled  as  the  sum  of  Po  analog  pulsed  sinusoids 
in  zero-mean,  normally  distributed,  independent  noise,  T}(t) 

P. 

x(t)  =  v{t)  +  g{t)  =  cos(27rFpt  -h  -t-  rj{t)  (4.1) 

p=i 

Here,  Fp  is  the  analog  frequency,  4>°  is  the  phase,  and  6°  is  the  amplitude  of  the  p*'*  sinu¬ 
soid.  This  analog  signal  is  passed  through  a  34‘*  order  Chebyshev  bandpass  antialiasing 
filter  with  critical  frequencies  as  shown  in  Table  4.1  (16).  This  filter  reduces  the  effects  of 
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Figure  4.1  EW  Digital  Receiver  Architecture:  K  =  32  Bandpass  Filters;  i?  =  16  Deci¬ 
mation  Rate;  8-bit  ADC  with  sample  rate  f,  =  3GHz;  Chebyshev  Antialiasing  Filter 
with  1.5  GHz  Bandwidth. 


out-of-band  interfering  signals  and  noise  while  passing  signals  within  a  specific  frequency 
range  for  digitization  (56:2).  For  bandpass  sampling  of  real  signals,  the  1.5  GHz  oper¬ 
ational  bandwidth,  (BW),  of  this  filter  is  related  to  the  sample  rate,  /,,  of  the  ADC  as 
2BW  <f,<  ^BW  (84:125). 

An  analog  signal  paissing  through  this  filter  is  sampled  by  the  eight  bit  ADC  at  a 
sample  rate  of  /,  =  3  x  10®  samples  per  second.  The  resulting  digital  signal,  xfm],  becomes 
(64:10) 

Po 

H  cos(27r/pTn  -(-  -b  rjlrn]  (4.2) 

P=l 

where  the  digital  frequency,  fp,  is  related  to  the  analog  frequency,  Fp,  via  the  transforma¬ 
tion  fp  =  Fp/fs-  Thus,  the  analog  frequency  range  l.bGHz  <  Fp  <  3GHz  is  inversely 
mapped  to  the  digital  range  0  <  fp  <  .5.  Finally,  the  noise,  7j[m],  is  zero  mean  and 
normally  distributed  with  a  covariance  matrix  determined  by  the  characteristics  of  the 
antialiasing  filter. 

In  order  to  provide  for  broadband  instantaneous  frequency  coverage  and  allow  for 
simultaneous  signal  detection  and  estimation,  the  operational  bandw'idth  is  divided  into 
K  =  32  specific  frequency  bands  by  employing  a  set  of  bandpass  filters.  The  coefficients 
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Table  4.1  Filter  Critical  Frequencies 


Filter  Type 

Order 

Freq 

Gain 

Anti-alicising 

34 

1.4953(GHz)  and  3.047(GHz) 
1.5047(GHz)  and  2.953(GHz) 

-60dB 

-MB 

-60dB 

-MB 

Bandpass 

192 

A:/32il/32 

fc/32il/64 

-60dB 

-MB 

for  the  bandpass  filter,  are  complex  and  derived  from  a  real,  symmetric,  lowpass 

filter,  fio[n],  by 

/i*[n]  =  (4.3) 

This  prototype  filter,  ho[n],  is  an  N  =  192  tap,  real,  symmetric  filter  designed  using  the 
McClellan-Ptirks  algorithm  to  meet  the  frequency  characteristics  given  in  Table  4.1.  Since 
^ ,  each  channel  of  the  filter  bank  will  be  a  bandpass  filter  with 
\Hk{e^'^'’^)\  =  0  for  \f  -  fk\  >  1/32.  For  a  real  signal,  x[m],  the  output  of  the  filter, 
yji,[m],  will  be  complex  and  given  as  the  convolution  sum 

N-l  K-lNo-l 

^  K[n]x[Tn  -  =  E  E  ho[l  +  nK]x[m  -  I  -  (4.4) 

n=0  /=0  n=0 

where  N  =  KNo  for  integer  No-  As  shown  in  Appendix  F,  ytlm]  can  be  interpreted 
as  the  Short  Time  Fourier  Transform  (STFT)  of  x\m]  as  seen  by  a  causal  finite  window 
q[n]  =  ho[N  —  1  —  n]  and  evaluated  at  u;  =  2-!Tk/K.  Conversely,  a  real  output,  ^*[771],  can 
be  obtained  by  adding  the  outputs  from  channels  k  and  K  —  1  —  k.  In  either  caise,  since 
the  spectrum  will  be  band-limited,  the  output  of  each  filter  can  be  translated  to  standard 
frequency  range  via  a  set  of  modulators  so  that 

fijkM  =  (4-5) 

This  demodulation  allows  for  a  standard  set  of  signal  detection  and  parameter  extraction 
techniques  to  be  developed  for  each  channel. 
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For  EW  applications,  the  speed  at  which  the  sampled  data  can  be  processed  is  much 
slower  than  the  data  rate  of  the  signals.  In  order  to  compensate,  after  demodulation,  the 
outputs  of  each  channel  can  be  decimated  by  an  integer  factor,  R,  so  that  v'l[m\  —  ut[i?m]. 
Consequently,  the  spectrum  of  t3*[m]  can  be  written  as(64:102-102) 

(4.6) 

This  decimation  process  serves  to  scale  the  frequency  spectrum  of  u*  [m] . 

After  decimation,  the  output  of  each  channel  is  p2issed  to  a  parameter  encoder  which 
estimates  the  parameters,  such  as  amplitude,  frequency  and  phase,  of  a  pulsed  sinusoid 
within  the  channel  and  outputs  a  Pulse  Discripter  Word  (PDW)  containing  these  estimates. 
These  estimates,  in  turn,  provide  the  data  needed  for  a  classification  system  to  determine 
the  type  of  radar  emitting  the  pulse.  Before  constructing  the  parameter  encoder,  the 
output  data  must  first  be  accurately  modeled. 

2.1  Real  Data  Model.  Since  all  K  =  32  filters  are  derived  from  a  single 
prototype  and  the  output  of  each  channel  is  modulated  to  a  standard  frequency  band  prior 
to  decimation,  the  output  of  each  channel  will  be  the  same  as  the  output,  y[m],  of  a  real 
filter,  h[n]  with  a  bandwidth  of  1/16,  followed  by  an  i?-fold  decimator.  For  the  sampled 
signal,  x[m],  using  the  results  from  Chapter  III,  an  expression  for  the  decimated  output 
can  be  written  in  the  form 

p 

y[m]  =  s[i2m]  +  u;[i2m]  =  ^  ftp  cos{2nRfpm  +  (f>p)  +  u;[i?m]  (4.7) 

p=i 

The  decimator  has  effectively  decorrelated  the  output  noise  by  the  decimation  factor  R 
and  scaled  each  sinusoidal  frequency  to  Rfp.  To  prevent  aliasing  and  allow  for  accurate 
frequency  estimation,  R  <  .5K  =  16.  In  addition,  since  several  of  the  sinusoids  will  have 
been  greatly  attenuated  by  the  filter,  the  summation  is  now  only  over  P  where  P  <  Po- 
The  problem  now  is  to  estimate  the  amplitudes,  {bp},  phases,  {<j>p},  and  frequencies,  [fp], 
of  the  sinusoids  given  the  decimated  meaisurements. 
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Now  eissume  there  are  M  samples  of  the  decimated  steady  state  output  and  define 
/i  as  a  vector  of  filter  coefficients  by  /i  =  [/i[0], . . . ,  h[N  -  1]]^  and  the  vector  of  R  zeros  as 
Ofi  =  [0 . . .  0]^.  Using  these  definitions,  the  R{M  -  1)  +  N  by  M  filter  matrix,  H,  can  be 
written  as 

h  Or  ...  Oft 
Oft  h  ...  Oft 

Ofl  Ofi  •  •  •  h 

For  m  =  0 ...  A/  —  1,  a  vector  form  of  the  decimated  output  can  be  written  as 

y  =  H^x  =  +  H^tj  =  s  +  w  (4.9) 

The  decimated  noise  vector,  to,  is  a  zero-mean,  normally  distributed,  correlated  noise 
sequence  with  covariance  Kyj  =  cr^H^H.  Again,  using  the  results  of  Chapter  III,  define 
the  deterministic  signal  matrix  ,  A-r,  as  =  [AijAa]  where  the  M  by  P  matrices,  Ai  and 
A2,  are  defined  by 

[Ai],,p  =  cos(27r/^P(M-/  +  iV-l]) 

[A2];,,  =  -  sin(27r/,P[M  -  I  +  N  -  l])  (4.10) 

for  1  =  1 ...  M  and  p  =  1 . . .  P.  With  scaling  vectors  b,  =  [61  sin(0i) . .  .bp  sin(<j!)p)p  and 
be  =  [61  cos(<^i) . .  .bp  cos(0ft)]^,  a  vector  form  of  the  decimated  output  becomes 

y  =  Ai&c -I- A26, -1- to  =  A-ftb  +  to  (4.11) 

where  Tl  denotes  the  real  data  model.  Here  b^  =  tbe  2P  columns  of  Ar  are 

linearly  independent.  This  is  equivalent  to  viewing  y  as  a  set  of  observations  generated 
from  the  deterministic  system,  A^b,  and  corrupted  by  to.  As  such,  the  PDF  for  this  model 
becomes 

p(y;0)  =  [27r]-**^|FC2o|-®exp[-i(y-Aftb)^FC^i(y- A^b)]  (4.12) 
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where  6^  =  [6^,  f^].  As  in  Chapter  III,  the  problem  of  estimating  the  frequencies,  phases, 
and  amplitudes  has  been  changed  to  the  problem  of  estimating  the  deterministic  signal 
matrix  Atj  and  the  scaling  coefficients  be  and  b,.  The  development  of  this  model  for  real 
filtered  and  decimated  data  is  an  original  contribution  of  this  research. 

4.2.2  Complex  Data  Model.  Since  the  input  signal  is  real,  the  outputs  from  chan¬ 
nels  0, . . . ,  Y  —  1  contain  the  same  information  as  the  outputs  from  channels  ,  K  —  1. 
Thus,  an  alternative  model  for  the  data  can  be  constructed  by  forming  a  complex  model 
for  the  data  from  channels  0, . . . ,  Kjl  —  1.  In  addition,  since  each  bandpass  filter  is  formed 
by  modulating  the  prototype  filter  with  a  complex  exponential  and  each  filter  output  is 
demodulated  to  a  specific  frequency  band  prior  to  decimation,  it  is  possible  to  build  a  sin¬ 
gle  complex  model  for  each  channel  based  on  a  standard  model.  That  is,  with  i[Tn]  being 
the  sum  of  Po  real  sinusoids  in  noise  and  A[n]  representing  a  complex  filter  band- limited 
to  1/16,  the  decimated  output,  y[m],  of  any  given  channel  can  be  written  as 

jf[m]  =  l[/iTn]  -f-  w[Rm]  (4.13) 

Here  s[Tn]  and  w[m]  represent  the  convolution  of  the  real  sinusoids,  u[m],  and  noise,  77[m], 
with  the  complex  filter.  For  steady  state  conditions, 

1 

5[m]  =  r  ^  (4.14) 

2  p=i 

where  6ip  =  6°|H'(e’"'')l;  ‘t>\p  =  <&(a;p)-f  cAp-,  and  b^p  =  6°|H’(e->">’)|  while  (Ajp  = 

Since  the  filter  is  band-limited,  assuming  only  P  complex  sinusoids  are  within  the  filter 
passband,  the  complex  form  of  the  output  signal,  y[Tn],  becomes 

p 

y[m]  =  +  w[RTn]  (4.15) 

p=i 

The  decimator  has  effectively  decorrelated  the  output  noise  by  the  decimation  factor  R 
and  scaled  each  sinusoidal  frequency  to  Rfp.  To  prevent  aliasing  and  adlow  for  accurate 
frequency  estimation,  R  <  K  =  32.  Employing  the  complex  data  model  for  the  output  of 
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each  channel  yields  a  method  to  increase  the  decimation  rate,  i?,  by  a  factor  of  2  over  the 
real  data  model  without  losing  information  and  is  an  original  contribution  of  this  research. 

For  this  model,  the  complex  noise  output,  u}[Tn],  is  a  random  variable  which  must 
be  described  statistically.  That  is,  u;[i2Tn]  =  Wr[Rm]  jwi[Rm]  where  Wrlm]  and  lUilm] 
represent  the  convolution  of  the  real  noise  with  the  real  and  imaginary  components  of  the 
filter.  Assuming  there  are  M  samples  of  the  decimated  steady  state  output,  with  the  vectors 
of  filter  coeflBcients  defined  as  hr  —  [hr[0], . . . ,  hr[N  —  1]]^,  amd  hi  =  [hj[0], . . . ,  hi[N  —  1]]^, 
the  R{M  —  1)  +  N  hy  M  filter  matrices,  Hr  and  Hi  can  be  written  as 


(4.16) 


Then,  in  vector  form  Wr  =  Hjri  where  Wr  =  [Wr[M  -  1] . . .  typ[0]]^  and  w,  =  HJt)  where 
Wi  =  - 1] . . .  ti)j[0]]^.  The  real,  augmented  noise  vector  =  [wj,  iwf  ]  is  a  normally 

distributed,  zero  mean  random  vector  with  covariance  matrix 

HjHr  HjHi 
HjHr  HjHi 

In  general,  Hj Hi  ^  Hj Hr  and  Hj Hi  ^  —Hj Hr  so  that  w  cannot  be  formally  represented 
as  a  complex  normally  distributed  random  vzu-iable  as  defined  in  Appendix  H. 

To  accurately  model  the  complex  data,  y,  rewrite  the  vector  in  terms  of  the  con¬ 
catenation  of  the  real  vector,  y,.,  and  the  imagineuy  vector,  y^,  as  y^  =  [y^;yf]  where 
Dr  =  [yr[M  -  1] . . .  yr[0]]^  and  y,-  =  [yi[M  -  1] . . .  y<[0]]^.  Expanding  yields 

F 

yrim]  =  cos{2-nRfpm)  —  bp  sin(27ri?/pm)  -t-  Wr[Rm] 

P=1 

P 

2/tM  =  &pSin(27ri?/pm) -1- 6' cos(27ri?/pm) -I- (4.18) 

p=l 


(4.17) 
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As  with  the  real  data  model,  =  6p  cos(0p)  and  6’  =  6psin(0p)  are  real  scaling  coefficients. 
Finally,  with  Ai  and  A2  defined  be  the  M  by  P  real  signal  matrices  of  Equation  4.10  then 


y  = 

Vr 

_ 

Ai 

A2 

be 

Wr 

Vi 

— A2 

Ai 

bs 

Wi 

=  Ac  b  +  w 


(4.19) 


where  Ac  is  the  deterministic  signal  matrix  for  the  complex  model.  Here,  y  and  w  6 
while  Ac  €  The  PDF  of  y  becomes 


p{y\9)  =  [2n]  ^\Kw\  ^  exp[--{y  -  Acbf  Ky}{y  -  Acb)] 


(4.20) 


This  equation  accurately  models  complex  data  derived  from  passing  a  real  signal  through  a 
complex  filter  and  is  an  original  contribution  of  this  research.  As  with  the  real  data  model, 
the  problem  of  estimating  the  frequencies,  phases,  and  amplitudes  has  been  changed  to 
the  problem  of  estimating  the  signal  matrix  A  and  the  scaling  coefficients  b,.  and  b,. 


4.2.3  Receiver  Implementation.  In  order  to  exploit  the  awlvantages  of  the  dec¬ 
imation  process,  the  digital  EW  receiver  described  above  is  implemented  as  a  multirate, 
nonmaximally  decimated,  polyphase,  UDFT  filter  bank  where,  in  general,  the  number  of 
channels  K,  is  an  integer  multiple  of  the  decimation  rate  R.  This  unique  implementation 
allows  every  operation  after  the  ADC  to  be  performed  at  a  rate  of  1/R  the  rate  suggested 
by  Figme  4.1. 

To  accomplish  this  data  rate  reduction,  the  prototype  filter,  /loH  must  decomposed 
into  K  polyphase  components  E/dz)  as  (92) 

K-l 

Ho{z)  =  S  z-‘Edz^)  (4.21) 

(=0 

Here,  Ei{z^)  =  ho[nK  +  and  the  integer  No,  defined  by  KNo  =  N,  is  the 

length  of  the  polyphase  filter.  As  shown  in  Figure  4.2,  with  the  Inverse  Discrete  Fourier 
Transform  (IDFT)  matrix,  Wiopj.,  defined  by  ]}A^iDFT\k,i  —  the  output  of  the 
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Figxire  4.2  UDFT  Filter  Bank  Polyphase  Implementation 


channel  is  given  as 


K-1 


yk[m]  =  ^  t,[m]exp{j2i:lk/K} 


(4.22) 


1=0 


where  t([m]  is  the  output  of  the  I*’'  polyphase  component.  Thus,  the  z-transform  of  y*[m] 
becomes  ^  ^ 

Y,{z)  =  X;  z-‘exp{j2nkl/K}Et{z^)X{z)  (4.23) 


i=0 


so  that  the  transfer  function  for  the  kth  channel  can  be  written  as 


Hk{z)  =  ^  =  ^{zexp{-j2irk/K}r‘E,{z^)  (4.24) 

,=o 

Since  {z  exp{-j2Kk/K})^  =  evaluating  Equation  4.24  at  z  =  z  exp{-j27rfc/iC},  yields 


K-l 


Hk{z)  =  Ho{zexp{-j2Trk/K}) 


(4.25) 


This  shows  is  the  output  of  a  bandpass  filter  centered  at  u;  =  2T:k/K. 

In  order  to  reduce  the  speed  at  which  the  filter,  IDFT  matrix  and  modulators  operate, 
the  R-fold  decimators  C2in  be  translated  from  the  output  of  the  modulators  to  the  input 
of  the  polyphase  filters  as  shown  in  Figure  4.3.  Letting  be  the  decimated  output  of 
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Figtire  4.3  Nonmaximally  Decimated  Polyphase  UDFT  Filter  Bank 
the  k^’'  channel,  then 


K-1 

(4.26) 

1=0 

Since  the  first  exponential  term  is  independent  of  R,  the  decimators  can  be  moved  to 
the  inputs  of  the  IDFT  matrix  to  allow  calculation  of  the  IDFT  and  modulators  at  the 
decimated  rate.  Decimating  the  output  of  the  polypheise  filter  gives  the  sequence 
sf[m]  =  ti[Rm]  so  that  the  z-transform  of  sf[Tn]  becomes 


=  i  I  (.-«exp  X  {^}) 


(4.27) 


Since  the  ratio  K/R  =  F  is  an  integer  and  exp{-'^^^— -}  =  1  for  all  integers  r  then 


Si{z)  =  E,(z^)i 


exp 


(4.28) 


This  is  equivalent  to  replacing  the  filter  Ei(z^)  with  Ei{z^)  and  moving  the  R  fold  dec- 
imator  to  the  firont  of  the  filter  bank  as  shown  in  Figure  4.3.  All  operations  in  the  filter 
bank  are  now  accomplished  at  1/ R^^  the  rate  of  the  input  data. 


In  terms  of  the  time  domain,  the  output  of  the  polyphase  component  can  be 
written  as 

{ho[KnlF  +  1],  for  n  =  pF  for  integer  p 

(4.29) 

0,  otherwise 

Since  N  =  KNo,  the  decimated  output  of  each  polyphase  component  becomes 

H  =  S  +  mK]x[Rm  -  I  -  nK]  (4.30) 

n=0 

The  IDFT  output  is  then 

K-l  N^-l 

yk[m]  =  nK]x[Rm  -  I  -  (4.31) 

1=0  n=0 

This  expression  is  equivalent  to  Equation  4.4  for  the  decimated  case  where  m  =  Rm. 

The  unique  implementation  of  a  nonmaximally  decimated  polyphaise  filter  bank  for 
R  =  FR  =z  32  consists  of  zero-padding  each  polyphase  component  filter  with  F  -  1  zeros 
and  decimating  by  R  the  input  across  all  K  channels  prior  to  taJdng  the  IDFT.  For  the 
real  data  model,  since  R  <  .5K,  the  maximum  rate  of  decimation  is  8.  Alternatively,  for 
the  complex  data  model,  since  R  <  K,  the  maiximum  rate  of  decimation  is  16.  Thus,  if 
the  accuracy  of  the  frequency  estimates  obtained  from  the  complex  data  model  are  equal 
to  those  obtained  from  the  real  data  model,  the  complex  data  model  should  be  employed. 

As  shown  in  Appendix  I,  this  aurchitecture  for  performing  channelization  via  a  non- 
majcimally  decimated  polyphase  filter  bank  is  mathematically  related  to  a  similar  method 
used  by  Rabiner  and  Crochiere  (55).  However,  the  anchitecture  derived  here  requires  only 
N  unique  filter  coefficients  to  produce  the  equivalent  STFT.  Conversely,  the  architecture 
proposed  by  Rabiner  and  Crochiere  (55)  requires  the  installation  of  K,  F-fold  expanders 
following  decimators  and  uses  F  subsets  of  R  unique  filters;  each  of  length  N/R,  for  a 
total  of  FN  filter  coefficients  to  produce  the  equivalent  STFT.  This  eirchitectural  design 
is  an  original  contribution  of  this  research  and  has  been  accepted  for  publication  (99).  In 
addition,  the  architecture  has  been  assigned  Serial  Number  08/816,951  by  the  U.S.  Patent 
and  Trademark  Office  and  given  a  filing  date  of  21  Jan  97. 
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4-3  Parameter  Estimation 

4-3.1  Estimation  Accuracy.  As  described  in  Chapter  III,  a  critical  step  in  the 
development  and  analysis  of  point  estimators  is  to  accurately  model  the  PDF  of  the  data. 
For  both  the  real  and  complex  cases,  the  data  model  was  shown  to  be  equivalent  to  viewing 
y  as  a.  set  of  observations  generated  from  the  deterministic  system  A6  corrupted  by  the 
zero  mean,  normally  distributed,  correlated  noise  vector  w. 


y  =  Aib  +  w 


(4.32) 


where  1  —  11  denotes  the  real  data  model  and  i  =  C  denotes  the  complex  data  model. 
The  problem  of  estimating  the  frequencies,  ph2ises,  and  amplitudes  was  changed  to  the 
problem  of  estimating  the  signal  matrix,  Aj,  and  the  scaling  coeflBcient  vector,  6,  from  the 
measurements  in  y.  In  addition,  for  both  models,  the  PDF  of  the  measurement  vector  had 
the  form 

=  [27r]"'^‘|i^t«r  ®exp[-i(y  -  AibfK^{y  -  Ai6)]  (4.33) 


where  Mi  =  -.5M  for  the  real  model  and  Mi  =  M  for  the  complex  model.  For  both 
models,  as  shown  in  Chapter  III,  the  CRLB,  which  provides  a  limit  on  the  accuracy  of  any 
unbiased  parameter  estimator,  can  be  calculated  from 


(dm 


y 


dBi  J 


[K: 


1-1 


w\ 


dm 


y 


[  ddj 


>[0] 


(4.34) 


Figure  4.4  shows  the  CRLB  for  the  frequency  of  a  single  sinusoid  using  both  the  real 
and  complex  models  for  data  obtained  from  channel  six  prior  to  decimation.  As  shown  in 
the  figure,  the  CRLB  for  the  frequency  of  a  sinusoid  within  the  filter  peissband  is  nearly 
the  same  for  either  model.  Thus,  there  should  be  virtually  no  loss  in  estimation  accuracy 
attributable  to  the  use  of  the  complex  data  model  instead  of  the  real  data  model.  However, 
since  the  complex  data  can  be  decimated  at  twice  the  rate  of  the  real  data  without  alicising, 
the  complex  data  model  of  Equation  4.19  will  be  employed  to  develop  point  estimates  of 
the  sinusoidal  parameters. 
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CRLB  for  Filtered  Sinusoid 


CBLB  for  Filtered  Sinusoid 


Fignre  4.4  CRLB  for  Real  and  Complex  Data  Models:  Variance  bounded  as  func¬ 
tion  of  frequency  and  SNR  over  filter  passband  for  undecimated  signal  of  block 
length  M  =  32.  Filter  characteristics:  Center  Frequency:  3/16;  Bandwidth:  1/32; 
Length:  N  =  192. 
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4-3.2  ML  Estimation.  As  shown  in  Chapter  III,  parameter  estimators,  whose 
estimates  of  the  sinusoidal  parameters,  6  =  [6^,  f^]^,  attain  the  CRLB,  can  be  developed 
by  maximizing  the  parameter  likelihood  function.  For  the  complex  data  model,  the  ML 
estimate  for  9  can  be  obtained  by  minimizing  the  following  objective  function,  with  respect 
to  the  parameters  in  d\ 

J{e)  =  [y-  Ac6]^  [y  -  \cb]  (4.35) 

For  a  fixed  ML  estimate  of  /,  the  ML  estimator  for  b  is  unique  and  found  as  (22),  (40) 

biML)  =  [A^Ki}Ac]~'  KlKi^^y  (4.36) 

The  ML  estimate  for  /,  on  the  other  hand,  minimizes  the  following  objective  function  (40): 

•^(Z)  ~  y^Ky}y  —  y^ Ky} Ac  [Ajii’^i/Ac]  AcKy^y  (‘^•37) 

Again,  as  shown  in  Chapter  III,  this  objective  function,  termed  the  ML  frequency  objective 
function,  is  a  highly  nonlinear  function  with  respect  to  /.  Direct  minimization  involved  a 
computationally  intensive  search  in  the  P  dimensional  frequency  domain  based  on  gradient 
search  algorithms.  In  general,  though  the  method  of  gradient  seeirch  can  provide  accurate 
frequency  estimates,  this  technique  usually  requires  too  many  iterations  to  converge,  and 
is  too  sensitive  to  the  initial  estimates  for  this  EW  application.  Alternatively,  a  simpler 
estimator  for  the  frequencies  can  be  developed  by  employing  LP  modehng  techniques  for 
the  complex  data  model. 

4-3.3  LP  Modeling.  In  general,  a  complex  signal,  s[m],  consisting  of  the  sum  of 
P  complex  sinusoids  can  be  represented  as  the  sum  of  P  complex  modes  (73:484-485) 

p  p 

l[m]  =  Y.  =  Y  +  js,[m]  (4.38) 

*=1  *=i 

Here,  z*  =  while  Sr[m]  and  3i[m]  are  the  real  and  imaginary  components  of  s[m].  A 

P**  order  complex  polynomial,  Ao{z),  which  incorporates  these  modes  can  be  constructed 
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as 

Po  P, 

Ao{z)  =  ao[0]  ]][(!  -  =  '^a[p]z~’’  (4.39) 

fc=l  p=0 

where  the  LP  coeflBcients,  a[p],  are  complex  quantities  given  by  a[p]  =  a^lp]  +  jai[p].  Thus, 
=  0  for  each  of  the  P  frequencies  present  in  s[Tn].  Since  each  mode  is  a  zero  of 
A{z),  the  P  frequencies  present  in  s[m]  can  be  found  as  roots  to  A{z)  and  the  corresponding 
time  domain  expression  becomes  the  complex  LP  equation. 

p 

^  d[p]s(m  —  p]  =  0  (4.40) 

p=0 

For  this  equation  to  be  true  for  M  data  samples,  0  <  m  <  M  —  1,  both  the  real  and 
imaginary  components  must  be  zero.  That  is 

p 

ar\p\sr[Tn  -  p]  -  a<[p]s4m  -  p]  =  0 

p=0 

p 

Y  ar[p]«i[Tn  -  p]  +  Oj[p]s,.(Tn  -  p]  =  0  (4.41) 

p=0 

To  construct  a  matrix  form  of  this  equation  for  m  =  P . . .  M  -  1,  define  the  LP  coefficient 
vectors,  and  Ci,  as  Or  =  [<ir[0] . . .  ar[P]]^  and  Oj  =  [ai[0] . . .  a^P]]^  and  the  component 
signal  matrices,  Sr  and  Si,  as 


=  s,[Af  +  l-k-l] 

(4.42) 

+  1-  k-l] 

(4.43) 

for  fc  =  1 . . .  M  —  P  and  I  =  1 ...  P.  Using  these  definitions,  the  complex  LP  general  linear 
model  becomes 

_  _  f  5.  -5, 

5,  5. 


Or 

tti 


=  Sac  =  0 


(4.44) 
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From  the  complex  data  model,  since  y[Tn]  =  s[m]  +  w[Tn],  in  terms  of  the  measurements, 
the  complex  LP  general  linear  model  can  be  written  eis 


1 

..  _ _J 

<3 

1 

Or 

1 

1 - 

.  J 

e> 

- 1 

(4.45) 


Here,  the  matrices,  Yr,  Vi,  and  Wr  are  defined  in  a  manner  similcir  to  Sr  and  Si  while 
Zr  and  Zi  can  be  considered  as  the  real  and  imaginary  error  vectors.  Finally,  define  Ar  as 
the  M  —  P  hy  M  matrix  of  real  coefficients 


* 

ar[k  —  1]  ioT  I  <  k  <  I  +  P 
0  otherwise 


and  Ai  as  the  associated  matrix  of  imaginary  coefficients 


a,[fc  —  1] 
0 


(or  I  <  k  <  I  +  P 
otherwise 


(4.46) 


(4.47) 


for  /  =  1 . . .  M  -  P.  The  vector  of  errors,  z,  can  be  written  as 


Zr 

4’’ 

1 

1 

Wr 

Zi 

_ 1 

Al 

Wi 

(4.48) 


Here,  Ac  is  the  matrix  of  coefficients  for  the  complex  LP  model.  As  a  linear  transformation 
of  tn,  a  zero  mean,  normally  distributed  random  vector,  z  becomes  a  zero  mean,  normally 
distributed  random  vector  with  covariance  matrix 


Kz  —  A^  KyjAc  (4.49) 

Premutiplying  Equation  4.45  by  G(ac),  the  Cholesky  decomposition  of  yields  a  new 
form  of  the  complex  LP  general  linear  model 

G(ac)Voc  =  V(oc)oc  =  e(oc)  (4.50) 
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Here,  e{ac)  is  a  zero  mean,  normally  distributed  random  vector  with  covariance  matrix 
Ke(ac)  ~  estimation  of  the  P  frequencies  in  s[Tn]  has  been  receist  as  the  estimation 

of  the  2P  +  2  LP  coefficients  using  the  LP  general  linear  model  derived  in  Equation  4.50. 
The  number  of  LP  coefficients  to  be  estimated  can  be  reduced  further  by  imposing  sinu¬ 
soidal  constraints  on  the  coefficients. 

As  shown  in  Appendix  C,  for  P  complex  sinusoids,  the  LP  coefficients  are  complex 
symmetric  with  d[p]  =  a[P-p\*.  Thus  only  P-1-1  coefficients  are  necessary  to  determine  ac 
completely.  As  in  Chapter  III,  let  a  be  the  set  of  P-t-1  real  coefficients,  q  =  [a[0] . . .  a[P]]^, 
With  P  being  an  even  number,  the  symmetry  constraints  can  be  imposed  by  defining  the 
vector  a  as 

a[0]  =  afi[L] 

a[fc -I- 1]  =  aftffc]  forfc  =  0...P-l  (4-51) 

a[L  -1-  /  -f- 1]  =  a/[/]  for  Z  =  0  ...  L  -  1 

where  L  =  P/2.  Alternatively,  if  P  is  an  odd  number,  let  L  =  {P  -  l)/2.  The  symmetry 
constraints  can  be  imposed  by  defining  the  vector  a  as 

Q![0]  =  anlL] 

a[A: -t- 1]  =  for  fc  =  0 . . .  L  -  1  (4.52) 

a{L  -I-  Z  -f- 1]  =  a/[Z]  for  Z  =  0 . . .  L 

In  either  case,  the  LP  coefficient  vector  aj  =  [o^,  of]  can  be  derived  from  a  as  ac  =  Ba 
where  B  is  the  matrix  relating  the  P  -I-  1  unique  coefficients  in  a.  to  ac- 

Imposing  these  constraints  on  the  complex  LP  general  linear  model  yields 

G{ac)YBa  =  G{a)Yca  =  e{a)  (4.53) 

The  seeirch  for  the  LP  coefficients,  contained  in  Oc,  has  been  reduced  to  the  identification 
of  the  P-l-1  constrained  coefficients  contained  in  Q.  Here,  Tc  is  an  M  —  2P  by  P-l-1  matrix 
of  rank  P  +  1.  Now  let  be  the  first  column  of  Yc  and  let  Y^  be  an  M  —  2P  by  P  matrix 
of  the  remaining  columns  of  Yc-  By  constraining  a[0]  =  1  and  defining  y^iot)  —  G{a)y^ 
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and  Fclo:)  =  a  final  form  of  the  complex  LP  general  linear  model  becomes 

y^{a)  =  -Yc{a)aa  +  e(a)  (4.54) 

Equation  4.54  represents  the  true  general  linear  model  relating  the  LP  coefficients  to  the 
frequencies  present  in  P  complex  sinusoids  in  noise  and  is  an  original  contribution  of  this 
research  and  has  been  submitted  for  publication  (97).  The  problem  now  is  to  estimate  the 
P  unique  LP  parameters  in  Qq  given  the  M  observations  with  M  >  P. 

4-3.4  LP  Model-ML  Frequency  Estimation  Relationship.  As  described  in  Chap¬ 
ter  III,  there  exists  a  relationship  between  the  complex  LP  general  linear  model  of  Equa¬ 
tion  4.54  and  the  ML  frequency  objective  function  of  Equation  4.37.  First,  assume  that  y 
has  length  2M  >  2P  and  has  2P  linearly  independent  columns.  From  Equation  4.19, 
since  Ac  is  an  2M  by  2P  matrix  with  2P  linearly  independent  columns,  the  column  vec¬ 
tors  of  Ac,  denoted  A<,  for  i  =  1 . . .  2P,  span  a  2M  dimensional  subspace,  Vj,  of  the  real 
vectors,  .  As  such,  there  exists  a  2M  —  2P  dimensional  orthogonal  subspace,  denoted 
V2,  which  is  spaimed  by  the  vectors,  Oj  for  j  =  1 . . .  2M  -  2F,  such  that  if  '0  6  Vj  and 
7  €  Vj  then  =  0  (83:132-152).  Now  consider  the  matrix  Ac  given  by  Equation  4.48. 
From  Equation  4.41,  for  the  complex  sinusoids,  A^Ac  —  [0]-  The  2M  —  2P  columns  of 
A  cire  orthogonal  to  the  2P  columns  of  Ac-  Furthermore,  since  the  columns  of  Ac  are 
linearly  independent,  they  form  a  basis  for  V^,  the  subspace  orthogonal  to  Vj.  Employing 
the  same  steps  as  in  Chapter  III,  minimization  of  J{f)  with  respect  to  /  can  be  written 
as  a  minimization  of  J{Ac)  with  respect  to  the  basis  vectors  in  .4c  where 

J{Ac)  =  y^  Ac{Ac  KxvAc)~^  A^y  (4.55) 

Finally,  since  each  column  of  Ac  is  derived  from  the  LP  coefficient  vector,  Oc,  then  J{Ac) 
can  be  written  cis 

J (ac)  —  {Al^ K%i} Ac)  ^Y ac  (4.56) 
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Employing  the  complex  sinusoidal  constraints  via  the  matrix  B  and  taking  the  Cholesky 
decomposition  the  inverse  of  A^Kw^c  yields 

J(a)  =  a'^Yj  G{afG{a)Yca  (4.57) 

By  defining  the  real  and  imaginary  error  vectors  as  er(a)  =  [er[M  -  1]  ...er[P]]^  and 
ej(a)  =  [ej[M  —  1] . . .  and  constraining  a[0]  =  1  yields 

e(a)  =  G{a)Yca  =  y^{a)  +  Y^{ci)cio  (4.58) 

where  e(a)^  =  [er(Q)^,  ei(a)^]  represents  the  concatenation  of  the  real  and  imaginary 
error  vectors.  This  equation  is  identical  to  the  complex  LP  generad  lineaur  model  derived 
in  Chapter  III.  By  properly  incorporating  the  effects  of  the  filter  coefficients  and  the  LP 
coefficients  in  developing  the  LP  general  linear  model,  this  derivation  shows  the  exact 
relationship  between  the  LP  coefficients  and  ML  complex  fi-equency  estimates.  The  set  of 
LP  coefficients  which  minimize  the  squared  error  defined  by  the  LP  general  linear  model 
will  provide  ML  estimates  of  the  frequencies.  Furthermore,  via  the  invariance  of  ML 
estimators,  the  LP  coefficients  which  minimize  the  squared  error  aure  the  ML  estimates  of 
the  coefficients. 

4-3.5  LP  Coefficient  Estimation.  Ideally,  the  optimal  estimator  for  the  LP  coef¬ 
ficients  would  minimize  J(q)  SO  that  the  resulting  frequencies  would  be  ML  estimates.  As 
discussed  in  Chapter  III,  minimization  of  J(a)  with  respect  to  a  is  a  nonlineair  optimiza¬ 
tion  problem.  Direct  minimization  involved  a  computationadly  intensive  search  in  the  P 
dimensional  coefficient  domain  baised  on  gradient  descent  algorithms.  In  generad,  though 
the  method  of  gradient  descent  can  provide  accurate  LP  coefficient,  amd  consequently,  fre¬ 
quency  estimates,  this  technique  usually  requires  too  many  iterations  to  converge  to  an 
answer  and  is  too  sensitive  to  the  initial  estimates  for  this  EW  application. 

Alternatively,  since  Equation  4.54,  has  the  same  form  as  the  LP  general  linear  model 
derived  in  Chapter  III,  a  fixed  point  estimator  can  be  constructed  to  estimate  a  from  the 
complex  data.  As  shown  in  Chapter  III,  the  ILS  and  ITLS  fixed  point  estimators  were 
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found  to  produce  estimates  nearly  as  accurate  as  those  attained  by  the  direct  minimization 
of  J{ci).  However,  these  estimators  took  fewer  iterations  to  converge  and  were  less  sensitive 
to  initial  estimates.  Since  the  ILS  estimator  was  found  to  produce  estimates  as  least  as 
accurate  as  the  ITLS  estimator  while  being  less  computationally  intense,  this  estimator 
will  be  used  to  provide  frequency  estimates  of  the  decimated  output  of  the  EW  receiver. 

Prom  the  system  model,  with  a°  =  Op,  let  aj,  be  the  estimate  of  the  LP  coeffi¬ 
cients.  Employing  the  ILS  fixed  point  algorithm  of  Chapter  III  yields 

=  -  [y/(a‘ )y,(Q‘ )] y/ (a*  )y(a‘ )  (4.59) 

When  =  a*,  the  error  vector,  e(a‘)  is  orthogonal  to  the  columns  of  yc(Q‘)  and 

a'g  =  ZiLsict'o)-  A  fixed  point  has  been  reached  and  the  estimate  of  the  LP  coefficients 
becomes  =  aj,. 

Figure  4.5  and  Figure  4.6  show  the  accuracy  of  the  true  and  the  approximated  ILS 
estimator,  in  relation  to  the  frequency  estimation  technique  employed  in  the  EW  receiver, 
for  estimating  the  frequency  of  a  single  sinusoid  at  the  filter  output.  As  shown  in  these 
figures,  the  ILS  estimator  attains  the  CRLB  at  an  SNR  approximately  5dB  lower  than 
the  current  phase-beised  WPA  estimator  and  maintains  this  accuracy  across  the  passband 
of  the  filter.  In  addition,  the  approximate  ILS  estimator,  constructed  with  Kyj  =  <7^7, 
performs  almost  as  well  8is  the  true  ILS  estimator.  This  indicates  the  effect  of  the  filter 
8ind  decimation  serve  to  decorrelate  the  data  enough  that  the  approximate  model  should 
be  adequate  for  estimating  the  frequency  of  a  single  filter  sinusoid. 

Figure  4.7  and  Figure  4.8  show  the  accuracy  of  the  true  and  the  approximated  ILS 
estimator  for  estimating  the-  frequency  of  a  two  sinusoids  at  the  filter  output.  At  the 
present  time,  the  EW  receiver  does  not  have  the  capability  to  estimate  the  frequencies 
of  two  time-coincident  sinusoids  within  the  filter  peissband.  As  shown  in  Figure  4.7a,  for 
two  sinusoids  with  frequencies  near  the  passband  frequencies,  the  frequency  estimates  do 
not  reach  the  CRLB  at  low  SNRs.  This  is  probably  due  to  the  severe  reduction  in  signal 
amplitude  occtunring  at  the  edge  of  the  filter.  Figure  4.7b  shows  the  estimation  accuracy 
for  two  closely  spaced  sinusoids.  Here,  the  decimated  frequency  sep^ation  of  df  =  .01  can 
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not  be  resolved  through  the  use  of  a  Fourier  Transform.  The  approximate  ILS  estimator 
appears  to  outperform  the  true  ILS  estimator  at  low  SNRs.  Close  analysis  of  the  figure 
shows  the  approximate  ILS  estimator  is  providing  biased  frequency  estimates  at  these 
low  SNRs.  Above  the  threshold  of  about  IdB,  both  estimators  perform  about  the  same. 
Finally,  Figure  4.8c  shows  the  accuracy  of  both  ILS  estimators  for  a  more  typical  set  of 
signals.  Here,  since  the  two  sinusoids  are  within  the  passband  and  are  adequately  sepeurated 
in  frequency,  both  estimators  attained  the  CRLB  at  less  than  — 5dB.  As  with  the  single 
sinusoid,  the  approximate  ILS  estimator  performed  as  well  as  the  true  ILS  estimator.  Thus, 
the  approximate  estimator  should  be  adequate  for  estimating  the  frequency  of  two  filtered 
sinusoids. 

4.3.6  Complex  Model  Approximation.  For  this  particular  prototype  filter,  ho[n], 
and  decimation  rate,  iZ  =  16,  experimental  analysis  shows  HT Hr  «  [0]  and  Hj Hr  «  Hj Hi. 
Using  these  approximations,  as  shown  in  Appendix  H,  the  PDF  of  the  complex  data  model 
can  be  written  in  the  form  of  a  complex  multivariate  Gaussian  PDF  (40:505-507) 

p{y^  exp  {-(y  -  Ab)^k:^{y  -  A6)}  (4.60) 

Here,  the  M  by  P  complex  matrix,  A  is  defined  by 

complex  vector  b  is  defined  by  bp  =  bpC^'^’’.  In  addition,  —  a^H^H  where  H  is  a. 
complex  R{M  —  l)  +  NhyM  filter  matrix  constructed  using  the  complex  filter  coeflficients 
in  Equation  4.8.  This  is  equivalent  to  viewing  y[m]  as  the  output  of  a  complex  filter  {^[n]} 
due  to  a  linear  sum  of  P  complex  sinusoids  in  normally  distributed,  independent,  complex 
noise,  ^[m].  Using  this  approximation,  equations  for  the  input  and  output  data  become 

=  I2p=i  bpe’^re^^^'A  ^[771]  and  y[m.]  =  I3p=i  bpe^'^’^e^^'^A  +  ,£,[771]  (4.61) 

where  ii;[m]  is  a  zero- mean,  complex  Gaussian  noise  sequence  with  a  covariance  matrix 
given  by  Kyj  =  a^H^ H.  This  approximation  allows  a  simpler  method  of  characterizing 
the  output  of  the  digital  receiver  and  is  an  original  contribution  of  this  research. 
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-10log10(MSE)  -10logl0(MSE) 


Frequency  Measurement  Accuracy:  SNR  -  OdB 


(a)  SNR  =  0  dB. 

Frequency  Measurement  AccuracytSNR  <•  -5d8 


(b)  SNR  =  -5  dB. 

Figure  4.6  ILS  Frequency  Estimation  Accuracy  Across  Bandwidth:  MSE  cal¬ 
culated  from  500  independent  realizations  of  block  length  M  =  32.  Ten  iterations 
allowed  for  convergence.  The  approximate  model  assumed  Kyj  =  <7^/  while  the  true 
model  used  correct  Kyj-  SNR  =  -101ogio(2(T^);  MSE  =  ^  5Zi=i(/i[i]  -  fi)  Sinu¬ 
soidal  Parameters:  [6f  =  l,0f  =  3x/8]. 


4-23 


-10log10(MSE)  ^  -10log10(MSE) 


Frequency  Measurement  Accuracy 


Frequency  Measurement  Accuracy 


Figure  4.7  ILS  Frequency  Estimation  Accuracy  for  Two  Difficult  Sinusoids: 

MSE  calculated  from  500  independent  realizations  of  block  length  M  =  32.  Ten  iter¬ 
ations  allowed  for  convergence.  The  approximate  model  assumed  Kw  =  while  the 
true  model  used  Kw  SNR  =  -101ogio(2cr2);  MSE  =  5Zp=i  Eiri(/pW  “  /?)• 
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Frequency  Measurement  Accuracy 


Figure  4.8  ILS  Frequency  Estimation  Accuracy  for  Two  Typical  Sinusoids: 

MSE  calculated  from  500  independent  realizations  of  block  length  M  =  32.  Ten  itera¬ 
tions  allowed  for  convergence.  The  approximate  model  assumed  Kw  =  while  the 
true  model  used  Kw  SNR  =  -101ogio(2£r^);  MSE  =  X)p=i  Ei^il/pW  “  /?)• 


4-3.7  Bounding  the  Estimation  Error.  As  shown  in  Chapter  III,  the  error  in  a 
point  estimate  of  the  LP  coefficient  and,  consequently,  the  frequencies  can  be  bounded  if 
the  variance  in  the  estimates  is  known  or  can  be  calculated.  From  Equation  3.165,  since  the 
complex  LP  general  linear  model  has  the  same  form  as  the  real  LP  general  lineeir  model,  an 
unbiased  point  estimate  of  the  variance  can  be  calculated  directly  from  the  meaisurements 
as 

(4.62) 

Figmre  4.9  shows  the  average  acciuracy  of  the  point  estimates  of  the  LP  coefficient  variance 
for  one  and  two  filtered  sinusoids.  These  results  show  the  point  estimates  provided  by 
Equation  4.62  aire,  on  average,  very  close  to  the  true  values  of  the  variance. 

The  accuracy  of  the  variance  estimates  indicates  that  confidence  intervals  for  the 
frequencies  can  be  established  directly  from  the  variance  estimates  using  Equation  3.148 
from  Chapter  III.  That  is,  once  Kcto  been  estimated,  the  variance  in  the  frequency 
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estimate  can  be  calculated  from 


y{h}  = 


9pi^o) 
da 


'gp(Qo)' 
.  dao  . 


(4.63) 


where  gp{)  is  the  function  relating  the  LP  coefficients  directly  to  the  frequencies.  In  partic¬ 
ular,  for  a  single  complex  sinusoid  derived  from  the  constrained  LP  coefficients,  gp{)  and 
F{}  take  the  form 

A  = --tan~^{a<[0]};  ^{/i}  «  7r"^(l  +  ai[0]^)"^V"{ai[0]}  (4.64) 

TT 


Figure  4.10  shows  results  of  using  this  equation  to  establish  confidence  intervals  for  the 
frequency  of  a  single  sinusoid  in  the  passband  of  filter.  Thus,  from  the  point  estimates  of 
the  frequency,  /j,  and  standard  deviation,  the  true  frequency  was  within  one,  two  and 
three  standard  deviations  of  the  estimate  about  65%,  95%  and  99%  of  the  time. 

For  an  EW  receiver,  knowledge  of  the  estimation  accuracy  can  improve  the  accuracy 
of  the  classifications.  Currently,  the  signal  descriptions  employed  by  typical  EW  classifi¬ 
cation  systems  consist  of  range  limit  descriptions  for  the  signal  peirameters.  These  range 
limits  are  usually  constructed  with  the  ‘worst-caise’  measurement  assumption  of  a  receiver 
operating  at  threshold.  This  assumption  forces  the  classification  system  to  extend  the 
range  limits  of  the  signal  models  which  induces  regions  in  which  the  ranges  overlap  within 
the  models.  This  overlap  in  ranges,  in  turn,  causes  multiple  identifications  to  be  made  for  a 
single  signal.  Providing  an  assessment  of  the  goodness  of  an  estimate,  along  wdth  the  point 
estimate  itself,  can  lead  to  improved  classification  accuracies  by  allowing  the  width  of  the 
classification  range  limits  to  be  compressed  or  expanded  to  match  the  actual  measurement 
accuracy.  Finally,  providing  an  estimate  of  the  ’goodness’  of  the  meaisurement  will  allow 
poor  measurements  to  be  discarded  prior  to  classification. 


4-3.8  Simplified  Single  Sinusoidal  Estimator.  Examination  of  the  simulations 
shows  the  effect  of  decimation  by  i?  =  16  is  to  decorrelated  the  noise  such  that  the  LS 
estimates  of  a  for  Kyj  «  <72/  are  eis  accurate  as  those  attained  using  the  true  noise 
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%  Time  True  Freq  With  K-STDs  of  Estimate  %  Time  True  Freq  With  K-STDs  of  Estimate 


Frequency  Confidence  Intervals 


Frequency  Confidence  Intervals 


Figtire  4.10  Frequency  Confidence  Intervals:  Confidence  intervals  established  at  SNR 
intervals  of  .5dB  and  frequency  intervals  .005  from  500  independent  realizations 
of  Af  =  32  samples  of  sinusoids  in  noise.  LP  coefllcients  estimated  by  ILS  algo¬ 
rithm.  Ten  iterations  allowed  convergence.  Confidence  interval:  |/p(i)  -  fp\  <  k&f 
for  k  =  1,2,3.  SNR  calculated  as  SNR  =  —  101og^o(2<r^). 

4-28 


covariance.  This  approximation  will  allow  an  extremely  simple  form  of  the  LS  estimator 
to  be  implemented  for  a  single  sinusoid. 

From  Equation  4.40,  with  P  =  1  and  a[0]  =  1  the  LS  fixed  point  estimator  for 
a[l]  =  a  =  can  be  written  as 


yf  Vi 


(4.65) 


Here  yi  =  [y[M-l] . . .  y[l]]^  and  yo  =  [y[M-2] . . .  y[0]]^  while  A  is  the  Af-P  by  M  matrix 
of  LP  coefficients  defined  in  a  manner  similar  to  the  A  matrix  of  Chapter  III.  Examination 
of  the  matrix  product  shows  AfAi  has  the  following  characteristics: 


f  2  if  k  =  l 


hi  if  /  =  A:  +  1 
a*  if  k  -  1  —  I 
0  otherwise 

V 


(4.66) 


Using  a  Cholesky  decomposition,  AfAi  =  RiRf  where  Pj  is  an  M  —  1  by  M  -  1  lower 
triangular  matrix  given  by 


=  [ft'  = 


if  m  =  n 

y  m 

a*/r(m-i),(m-i)  if  n  =  m  -  1 
0  otherwise 


(4.67) 


Expansion  yields 
and 


[d*]"*  "c^nn  where  the  real  quantity,  Cm„,  is  zero  for  m  >  n 


Cmn  — 


(-!)( 


m— f») 


[[nr.;'r’. 


(4.68) 


for  n  <  m.  Thus,  is  a  known  constant  derived  only  from  the  real  constants  defined  in 
fmm-  Defining  the  M  — 1  by  M  — 1  matrix  D  of  constants  so  that  dki  =  c^kCnh 
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then  the  matrix  inverse  calculations  can  be  written  as 


_  1  iVi  —  i 


n=:l 


-1  r  . 


kn 


i?.  =  [a:]*4,[a. 

nl 


X  M 


(4.69) 


Thus,  by  constructing  the  vectors  Vi  and  from  and  2/2  as  {vi}m  =  ^r{yi}m  and 
{■Utlm  =  a  simplified  estimator  for  fij  can  now  be  written  as 


ill 


.  Vi  Vo  _  v^Duj 

Vi 


v^Dvi 


(4.70) 


The  matrix  inverse  operation  has  been  replaced  by  a  simple  vector-matrix-vector  product 
operation.  This  efficient  representation  is  useful  when  the  number  of  data  samples,  M, 
to  be  processed  becomes  large.  The  development  of  this  simplified  estimator  for  a  single 
complex  sinusoid  is  an  original  contribution  of  this  research  and  hais  been  submitted  for 
publication  (96). 

4-4  Summary 

In  this  chapter,  the  ILS  fixed  point  estimation  technique  was  successfully  used  to 
estimate  the  frequencies  of  sinusoids  at  the  output  of  the  digital  EW  receiver  being  de¬ 
veloped  by  AFRL/WL  (16).  After  discussing  the  general  receiver  architectme,  models 
were  derived  for  the  real  and  complex  data  output  from  bandpass  filters  and  represent  an 
original  contribution  of  this  dissertation.  The  model  derived  for  the  complex  data  shows 
the  effective  decimation  rate  of  the  output  data  be  doubled  over  that  of  the  real  model 
without  a  loss  of  frequency  estimation  accuracy. 

A  new  method  to  efficiently  implement  the  receiver  as  a  nonmaximally  decimated, 
UDFT  polypheise  filter  bank  is  then  derived  based  on  the  complex  data  model.  This  deriva¬ 
tion  has  been  accepted  for  publication  and  has  been  assigned  Sericil  Number  08/816,951 
by  the  U.S.  Patent  and  Trademzirk  Office  (99). 

Based  on  this  architecture  and  the  complex  data  model,  a  complex  form  of  the 
LP  general  linear  model  is  developed  for  estimating  the  frequencies  of  P  real  sinusoids 
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residing  in  the  filter  passband.  This  new  form  of  the  LP  general  linear  model  is  an  original 
contribution  of  this  research  and  has  been  submitted  for  publication. 

Identification  of  the  LP  coefficients  is  then  shown  to  be  related  to  ML  frequency 
estimation  and  an  ILS  fixed  point  estimator  is  derived  for  estimating  the  coefficients. 
For  one  sinusoid,  simulations  indicate  the  ILS  estimator  attains  the  CRLB  for  frequency 
estimation  at  SNRs  significantly  lower  than  those  currently  employed  in  the  receiver.  The 
ILS  estimator  Wcis  then  successfully  used  to  estimate  the  frequencies  of  two  time-coincident 
sinusoids  within  the  filter  passband;  a  capability  which  currently  does  not  exist  in  the 
receiver. 

This  chapter  concluded  by  showing  the  PDF  of  the  complex  data  can  be  approxi¬ 
mately  modeled  as  a  complex  Gaussian  PDF.  Using  this  approximation,  a  simplified  fre¬ 
quency  estimator  of  a  single  sinusoid  was  derived  which  effectively  removed  the  requirement 
to  invert  an  M  —  1  square  complex  matrix.  This  new  single  frequency  estimator  for  com¬ 
plex  sinusoids  is  an  original  contribution  of  this  dissertation  and  is  currently  under  review 
for  publication. 

Finally,  in  addition  to  providing  the  Air  Force  with  a  complete  probabilistic  charac¬ 
terization  of  their  next  generation  digital  EW  receiver  system,  this  dissertation  derived  a 
new  method  of  estimating  the  frequencies  of  filtered  sinusoids.  The  frequency  estimators 
developed  as  a  result  of  this  research  will  greatly  extend  the  operational  envelope  of  the 
EW  receiver. 
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V.  Conclusion 


5. 1  Introduction 

The  dissertation  investigated  the  problem  of  estimating  the  parameters  of  real  and 
complex  filtered  sinusoids  in  noise.  In  Chapter  III,  the  general  theory  of  parameter  esti¬ 
mation  for  real  filtered  sinusoids  in  noise  was  developed  and  used  to  construct  accurate 
parameter  estimators.  The  resulting  analysis,  models,  and  estimators  derived  in  this  chap¬ 
ter  have  applications  across  the  spectrum  of  applied  science,  engineering  and  statistics.  In 
Chapter  IV,  the  results  of  this  research  were  applied  to  the  Air  Force’s  next-generation 
EW  receiver.  The  resulting  analysis,  models,  and  estimators  derived  in  this  chapter  led 
to  an  efficient  hardware  implementation  of  the  receiver  while  significantly  improving  the 
receiver’s  operational  capability. 

5.2  Theoretical  Contributions 

As  shown  in  Chapter  III,  the  first  contribution  of  this  research  was  a  mathematical 
model,  given  by  Equation  3.13,  for  any  system  required  to  estimate  the  amplitudes,  phases 
and  frequencies  of  real,  filtered  sinusoids.  Specifically,  this  dissertation  showed  that  the 
steady  state  output  of  a  filter,  due  to  an  input  consisting  of  a  linear  sum  of  P  real  sinusoids 
in  zeromean,  independent,  normally  distributed  noise,  can  be  modeled  as  the  output 
from  a  deterministic  system,  Ab,  corrupted  by  zero-mean,  normally  distributed  noise,  w, 
with  a  covEiriance  matrix,  Kyj,  determined  by  the  filter  cheiracteristics.  Estimation  of  the 
sinusoidal  parameters  was  recast  as  the  estimation  of  a  deterministic  system  firom  the  set 
of  measurements,  y,  embedded  in  colored  noise  and  a  bound  on  the  parameter  estimation 
accuracy  was  derived. 

Using  this  model,  a  new  set  of  ML  estimators  for  the  scaling  parameters,  b,  and 
frequencies,  /,  were  derived  w'hich  correctly  account  for  the  correlation  in  the  noise  due  to 
the  effects  of  the  filter.  This  dissertation  showed  the  ML  estimator  for  b,  given  in  Equa¬ 
tion  3.23,  can  be  found  via  a  simple  linear  regression  on  the  measurement  vector  y  once 
an  ML  estimate  of  the  frequencies,  /,  is  made.  The  ML  estimate  of  the  frequencies,  on  the 
other  hand,  is  independent  of  b  and  can  be  found  by  optimizing  the  ML  frequency  objec- 


5-1 


tive  function,  J{f),  of  Equation  3.25.  Since  this  objective  function  is  a  highly  nonlinear 
function  with  respect  to  f,  the  ML  frequency  estimator  derived  in  Equation  3.30  involved  a 
computationally  intensive  search  in  the  P  dimensional  frequency  beised  on  gradient  search 
algorithms.  Simulations  indicate  the  accuracy  of  these  ML  estimators  will  achieve  the  the¬ 
oretical  bounds  provided  the  correlation  in  the  noise,  due  to  the  filter,  is  correctly  modeled. 
Failure  to  incorporate  the  filter  effects  in  the  ML  model  leads  to  suboptimal  peirameter 
estimates.  The  main  drawback  in  using  these  estimators  for  an  EW  receiver  is  that  the 
gradient  search  algorithm  employed  to  provide  ML  frequency  estimates  usually  required 
too  many  iterations  to  converge  and  relied  on  an  accurate  initial  estimate. 

To  overcome  these  drawbacks,  the  problem  of  frequency  estimation  was  recast  as  the 
problem  of  estimating  the  coefficients  of  the  order  LP  polynomial.  After  describing  the 
relationship  between  the  LP  coefficients  and  the  sinusoids,  a  linear  model,  parameterized 
by  the  LP  coefficients,  was  derived  in  Equation  3.53.  This  model,  which  incorporates  the 
effects  of  both  the  filter  and  the  coefficients  upon  the  noise,  is  the  true  linear  model  relating 
the  LP  coefficients  to  the  filtered  sinusoids  in  noise  and  is  an  original  contribution  of  this 
research  (98). 

Based  on  this  general  linezw  model,  two  estimators,  based  on  fixed  point  theory,  were 
then  derived  for  estimating  the  LP  coefficients  and  represent  another  original  contribution 
of  this  reseairch.  The  ILS  estimator,  as  given  by  Equation  3.84,  was  based  on  an  iterative 
least  squares  solution  to  an  over-determined  system  of  equations.  The  ITLS  estimator, 
as  given  by  Equation  3.100,  was  beised  on  an  iterative  total  le2ist  squares  solution.  Simu¬ 
lations  indicate  both  methods  provide  near-minimum  variance,  unbiased  estimates  of  the 
LP  coefficients,  and  consequently,  the  sinusoidal  frequencies,  over  a  wide  range  of  SNRs. 
Fmrthermore,  each  algorithm  usually  took  less  than  ten  iterations  to  converge  emd  was 
relatively  insensitive  to  the  initial  estimate.  In  addition,  the  initial  coefficient  estimate 
can  be  obtained  directly  within  the  framework  of  the  algorithms;  no  additional  estimation 
routines  are  required  to  provide  the  initial  estimates.  All  other  factors  being  equal,  the 
ILS  method  should  be  preferred  over  the  ITLS  method  since  the  ITLS  method  requires  an 
SVD  of  an  M  —  2P  by  P  -)- 1  matrix  at  each  iteration. 
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This  dissertation  then  derived  the  exact  relationship  between  ML  frequency  estima¬ 
tion  of  the  filtered  sinusoids  and  ML  estimation  of  the  LP  coefficients.  Specifically,  the 
set  of  LP  coefficients  which  minimize  the  square  error  objective  function,  J(q),  defined 
in  Equation  3.124,  was  shown  to  provide  ML  frequency  estimates  and  vice  versa.  This 
original  contribution  led  directly  to  the  development  of  the  lEGD  estimator,  given  by 
Equation  3.127,  for  minimizing  J(a).  This  estimator  provides  ML  estimates  of  the  LP  co¬ 
efficients  and,  consequently,  the  frequencies  of  filtered  sinusoids,  by  employing  an  iterative 
exact  gradient  descent  algorithm  to  minimize  J{a).  Simulations  indicated  the  estimates 
found  via  the  lEGD  algorithm  do  minimize  the  LP  objective  function  thus  providing  ML 
estimates  of  the  LP  coefficients  and,  consequently,  the  frequencies.  However,  this  algo¬ 
rithm  usually  required  too  many  iterations  to  converge  and  was  too  sensitive  to  the  initial 
estimates  to  be  of  much  use  as  a  real-time  frequency  estimator. 

This  dissertation  then  proved  the  equivalence  between  the  IGLS  algorithm,  previously 
presented  as  a  method  of  minimizing  the  LP  objective  function,  J(a),  and  the  ILS  fixed 
point  estimator  of  Equation  3.84.  This  proof  is  an  original  contribution  of  this  research 
for  it  correctly  casts  the  IGLS  algorithm  as  a  fixed  point  estimator,  not  a  minimization 
algorithm. 

In  addition,  the  IQML  algorithm  Wcis  proven  to  be  exactly  equivalent  to  the  ITLS 
fixed  point  estimator  of  Equation  3.100.  This  is  another  original  contribution  of  this  disser¬ 
tation  for  it  correctly  casts  the  IQML  algorithm,  widely  accepted  cis  the  premier  method  of 
minimizing  J{ct),  as  a  fixed  point  estimator,  not  a  minimization  algorithm.  In  addition,  for 
the  simulations  completed,  the  ILS/IGLS  algorithm  actually  produced,  on  average,  more 
accurate  frequency  estimates  than  the  more  comphcated  ITLS/IQML  algorithm.  The 
IQML  algorithm  requires  an  eigenvalue  decomposition  ofaP-flbyP-l-1  square  matrix 
at  each  iteration  whereas  the  IGLS  algorithm  only  requires  the  inversion  of  a  P  square 
matrix.  Thus,  if  the  small  degradation  in  performance  over  the  lEGD  can  be  tolerated, 
the  ILS/IGLS  estimator  should  be  preferred  over  the  ITLS/IQML  estimator  for  providing 
point  estimates  of  the  LP  coefficients  and,  consequently,  the  sinusoidal  frequencies. 

Finally,  a  method  was  developed  for  bounding  the  estimation  error  of  the  LP  coeffi¬ 
cient  estimates,  and  consequently,  the  frequency  estimates,  b8ised  strictly  on  one  realization 
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of  the  measurement  vector.  Specifically,  as  shown  by  Equation  3.165  ,  this  dissertation 
proved  the  point  estimate  of  the  covariance  matrix  of  the  LP  coefficients  is  unbiased.  This 
derivation  is  an  original  contribution  of  this  research  and  allows  the  estimate  of  the  LP 
coefficient  variance  to  be  transformed  into  an  unbi2ised  estimate  of  the  frequency  covari¬ 
ance  matrix.  In  addition,  as  shown  in  Equation  3.147  and  Equation  3.148,  from  a  single 
estimate  of  the  variance  of  the  LP  coefficients  and  the  frequencies,  the  estimation  error 
can  bounded  and  confidence  intervals  constructed.  Simulations  indicate  these  estimates 
of  the  variance  can  be  used  to  gauge  the  accuracy  of  the  point  estimates  of  both  the  LP 
coefficients  and  the  frequencies.  This  gauge  can  then  be  employed  to  develop  confidence 
intervals  to  aid  any  decision  making  process  based  on  a  single  set  of  metisurements. 

5.3  Applied  Contributions 

In  this  dissertation,  a  new  architecture,  assigned  Serial  Number  08/816,951  by  the 
U.S.  Patent  and  Trademark  Office,  weis  derived  to  implement  a  a  nonmaximally  decimated 
UDFT  polyphase  filter  bank.  This  architecture  is  based  on  a  new  data  model  relating 
parameter  estimation  of  real  sinusoids  to  the  parameter  estimation  of  complex  sinusoids 
and  is  an  original  contribution  of  this  research  (99).  Specifically,  as  shown  in  Figure  4.3, 
this  new  data  model  transforms  the  estimation  of  the  parameters  of  real  sinusoids  in  real 
colored  noise  to  the  estimation  of  the  parameters  of  complex  sinusoids  in  complex  colored 
noise.  Estimation  of  the  real  sinusoidal  parameters  is  recast  as  the  estimation  of  the 
deterministic  linear  system,  Acb,  from  a  set  of  complex  measmements,  y  embedded  in 
colored  noise.  Simulations  for  the  EW  architecture  indicate  this  transformation  can  be 
attained  with  negligible  loss  in  estimation  accmracy  attributable  to  the  complex  model. 

In  addition,  this  new  data  model  shows  how  complex  data,  originating  from  a  sum 
of  real  sinusoids  in  real  noise  passing  through  a  complex  bandpass  filter,  can  be  decimated 
at  twice  the  rate  of  the  real  data  without  a  loss  in  parameter  estimation  accuracy.  This 
improvement  in  the  decimation  rate,  exploited  by  transferring  the  decimators  to  input  of 
the  filterbank  cis  shown  in  Figure  4.3,  allows  a  significant  reduction  in  the  speed  at  which 
the  filters,  IDFT  matrix,  modulator  and  parameter  encoder  must  operate. 
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Based  on  this  new  model  for  a  complex  representation  of  real  sinusoids,  a  new  set 
of  ML  estimators  for  the  scaling  parameters,  6,  and  frequencies,  /,  were  derived  which 
correctly  account  for  the  correlation  in  the  noise  due  to  the  effects  of  the  filter.  This 
dissertation  showed  the  ML  estimator  for  6,  given  in  Equation  4.36,  can  be  found  via 
a  simple  linear  regression  on  the  measurement  vector  y  once  an  ML  estimate  of  f  is 
made.  As  with  real  sinusoids,  the  ML  estimate  of  the  frequencies,  on  the  other  hand,  is 
independent  of  b  and  can  be  found  by  optimizing  the  ML  frequency  objective  function, 
J{f)  of  Equation  4.37  . 

Since  this  objective  function,  is  a  highly  nonlinear  frmction  with  respect  to  /,  the 
problem  of  frequency  estimation  was  recast  as  the  problem  of  estimating  the  coefficients 
of  the  order  complex  LP  polynomial  given  by  Equation  4.40.  After  describing  the 
relationship  between  the  LP  coefficients  and  the  complex  sinusoids,  a  linear  model,  pa¬ 
rameterized  by  the  LP  coefficients,  was  derived  in  Equation  4.54.  This  model,  which 
incorporates  the  eflFects  of  both  the  filter  and  the  coefficients  upon  the  noise,  is  the  true 
linear  model  relating  the  LP  coefficients  to  complex  filtered  sinusoids  in  noise  and  is  an 
original  contribution  of  this  research  (97). 

Identification  of  the  LP  coefficients  was  then  shown  to  be  related  to  ML  frequency 
estimation  and  an  ILS  fixed  point  estimator  was  derived  for  estimating  the  coefficients. 
For  one  sinusoid,  simulations  indicate  the  ILS  estimator  attadns  the  CRLB  for  frequency 
estimation  at  SNRs  significantly  lower  than  those  attained  by  the  estimation  algorithm  cur¬ 
rently  employed  in  the  receiver.  The  ILS  estimator  was  then  successfully  used  to  estimate 
the  frequencies  of  two  time-coincident  sinusoids  within  the  filter  passbaind;  a  capability 
which  currently  does  not  exist  in  the  receiver. 

Finally,  the  results  of  this  applied  research  showed  the  PDF  of  the  complex  data 
out  of  a  given  channel  of  the  EW  receiver  filterbank,  can  be  approximately  modeled  as  a 
complex  Gaussian  PDF.  Using  this  approximation,  a  simplified  frequency  estimator  for  a 
single  complex  sinusoid  was  derived  which  effectively  removed  the  requirement  to  invert  an 
M  —  1  square  complex  matrix.  This  new  single  frequency  estimator  for  complex  sinusoids 
is  an  original  contribution  of  this  dissertation  and  heis  been  submitted  for  review  (96). 
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5-4  Summary 

The  results  of  this  research,  whether  taken  individually  or  collectively,  represent 
a  major  contribution  to  the  theory  of  signal  processing  and  parameter  estimation.  In 
particular,  this  research  builds  the  bridge  connecting  sinusoidal  frequency  estimation  with 
LP  linear  system  modeling  eind  derives  the  connection  between  real  and  complex  sinusoidal 
parameter  estimation.  In  addition,  the  analysis,  models  and  estimators  constructed  as  a 
result  of  this  connection  will  significantly  improve  the  operational  envelope  of  the  Air 
Force’s  next  generation  EW  receiver. 
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Appendix  A.  Vector-Matrix  Differentiation 


This  appendix  derives  the  identities  involving  the  differentiation  of  matrix  products. 


A.l  Definitions 

Let  0  be  P  dimensional  vector  with  0  =  [^i . . .  0p\^ .  Let  x  be  Q  dimensional  vector 
with  X  =  [ii  . . .  XqY  where  each  element  of  x  is  a  function  of  6.  That  is  x,  =  gq{d).  The 
first  and  second  derivatives  of  x  with  respect  to  9i  and  dj  are  defined  as 


'  dx' 

1 

1 

[•Q 

d'^x 

9^Xq 

99  i. 

,  99,' 

99,99 j. 

q  ~  99,d9j 

(A.1) 


Let  U  he  &  K  by  L  dimensional  matrix  where  each  element  of  f7  is  a  function  of  9.  That 
is  ujfe,/  =  9k,i{0).  The  first  and  second  partial  derivatives  of  U  with  respect  to  and  9j  are 
defined  as 


'dU' 

9uk,t 

•  ai^u  ■ 

[99,  \ 

"  99,  ' 

99,a9j\ 

d9id9j 


(A.2) 


A. 2  Matrix- Vector  Product  Derivative 

Let  U  he  an  M  by  N  matrix  and  x  by  an  dimensional  vector.  Let  „  and  x„ 
be  functions  of  the  variables  9i  and  9j.  In  addition,  let  z  =  Ux.  Then 


dZm  _  dUm,n  _  ,  dx„ 

d9i  ~  d9i 


dU 


dz 


dOi  [d9i\ 


x  +  U 


dx 


[d9, 


(A.3) 


while  the  second  partial  derivatives  become 


_  _  ^ 

d9id9j  “  d9id9j 


Xn  +  Wm.n 


dx„  dUm,n  dx„ 

d9,d9j  ^  d9i  d9j  d9j  d9i 


a'^z 

•  a'^u  1 

X  +  U 

9^x 

+ 

au' 

'  dx ' 

+ 

au' 

dx' 

a9,a9j 

[99,a9j\ 

99,a9j, 

.99,  _ 

.99  j. 

.99  j. 

.99,. 

(A.4) 

(A.5) 
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A. 3  Matrix  Product  Derivative 


Let  U  and  V  be  L  by  K"  and  L  by  L  matrices  respectively.  Let  u*,;  and  r;;  be 
functions  of  6  and  define  the  matrix  C  zs  C  —  U^VV .  Then 

dc/g  n  X — '  ^ I  Ovi  rn  n  /  ^ 

"  EE  +  (A.6) 


d^Ck,n  _  d^'^kj  ,  dutj  dvi^rn 


dUk,l  dUm,n 


96,86, 

dUk.l  9vi^m 


■*■  fl/1  az)  azi  az)  “f"."  ■*" 


86,  86, 

dvi^m  ^m,n 


86,  86, 


'86,86, 


86,  86, 


8Uk,l  8Um,n  ,  dvi^m8u^^n  , 

86,'^^'"'  86,  a^p 


In  matrix  nomenclature,  the  partial  derivative  with  respect  to  6,  is 


dc  \8UY  t\8V]  -  rai7 

VU  +  \  —  \  U  +  U^V 


86,  [86, 


while  the  partial  derivative  with  respect  to  6,  and  6,  becomes 


86,86, 


86,86, 

8U 

T 

de,. 

8U- 

T 

I 

VV ?Z\v .mv\?L 


d&pi  id6,i  [d6,\  [86, 

Ur\_fL]u^U-m 

[a^paej  ^  [a^pj  [86, 
[d6,\  [86, \  [86,86,, 


A. 4  Inverse  Derivative 


Let  C  by  a  symmetric,  invertible  matrix  where  ezich  element  is  a  function  of  0j.  Then 
(40:73-75) 

dc-^  raci 

8di  ~  [aej  ^ 


a^c-i 

86i86j 


^1  [^1  c-‘  -  C-^  f-^1  c-  -  c-  f^l  r A  in 
86j  J  [86ir  ld6,86j\^  ^  [a^J  [  86j 


86i}  [  86 j 
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A. 5  Vector- Matrix- Vector  Product  Derivative 


Let  X  be  dimensional  vector  and  let  C  he  a  K  hy  K  positive  definite 
Furthermore,  let  the  scalar  J  be  defined  as  J  =  x^C~^x.  Then 


dJ 

ddp 

d^J 

de^dd. 


K  K 


Y'  y-  1  X  +X 

art  ^ 


n=l  m=l 


_ J.  +  J.  c-1 


dx„,dC^\  5xm  1  ax„ 


-2:n  + 


Qg  Cm,n  qq 


+ 

+ 


dx^ac-)„  ,  ,  ac-|„ax„ 

- -^n  +  Xm  o/T^n  +  - 


30,  30p 


30p30, 

-1 


30p  30, 


3xm^_i  3x„  dC^^dx 


CZ\^  +  +  x^Q 


3^x 

-1  -‘-n 


90,  90p 


30,  a0p 


”*  "‘■'‘90p30, 


In  vector-matrix  notation,  the  partial  derivative  with  respect  to  0p  becomes 


3J 

30, 


=  2 


3x 


iT 


30 


pj 


C  -f  X' 


.3C 


-1 


30„ 


The  partial  derivative  with  respect  to  0p  and  0,  can  be  written  as 


3V 

30p30, 


=  2 

+  2 


32x 


iT 


30p30,j 


C“*x  +  2 


3x 


tT  r 


30, 


« J 


3C 


-n 


30 


p  J 


X  -t-  X 


'  dx ' 

T 

■3C-1' 

[30pJ 

[  30, 

T 

\d^C 

-M 

X 

[30„30„. 

X 

X  -I-  2 


r3x 


30„ 


C 


-1 


3x 


«■ 
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matrix. 


(A.12) 


(A.13) 


(A.14) 


(A.15) 


Appendix  B.  Maximizing  ML  Objective  Functions 

This  appendix  derives  the  scalar  and  vector  identities  pertinent  to  optimization  of 
real  objective  functions  with  respect  to  real  and  complex  vectors.  These  identities  are  then 
used  to  obtain  ML  estimators  for  real  and  complex  data. 


B.l  Real  Data  ML  Maximization 

Let  b  =  [bi . . .  bq]^  be  a  vector  of  Q  real  parameters.  Let  a;  =  [ii  . . .  xq]^  be  a  vector 
of  constants  and  let  A  be  a  square  Q  by  Q  matrix  defined  by  =  a*;.  Finally,  let  J  be 
a  scalar  function  of  b  so  that  the  gradient  of  J  with  respect  to  b  is  defined  as 


VbJ  = 


dJ 


->T 


db 


•Q] 


(B.l) 


The  results  of  applying  this  gradient  to  vector  products  and  vector-matrix-vector  products 
is  given  in  Table  B.2.  With  6^  —  b^;  and  A  a  function  of  /  only,  the  likelihood 
function  is  defined  as 


p{y;d)  =  [27r]-'^lA:«;|-i  exp[-i(2/  -  AbfK^^iy  -  A6)]  (B.2) 

Talking  the  natural  logarithm  yields  the  log-likelihood  function  L{y,  9)  eis 

L{y;d)  =  -y  ln(27r)  -  i  In  -  1  [y  -  \bf  K^^[y  -  Ab]  (B.3) 

Assuming  Kw  is  known,  then  maximizing  L{y,  9)  is  the  same  as  minimizing 


J«(0)  =  [y-A6]’’A:^Ml/-Ab] 


(B.4) 


Expanding  yields 


Ja{9)  =  y^K^^y  -  26^A^K^'y  +  b’^A'^K^^Ab  (B.5) 
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Taking  the  gradient  with  respect  to  b  yields 


=  -2A^  +  2X^K^\\b  (B.6) 

Setting  equal  to  zero  and  solving  for  b  yields  the  ML  estimator  which  is  found  via  a  linear 
regression  on  y  provided  /  is  given. 

buL  =  X^K^'y  (B.7) 

Substitution  of  Bml  into  Jr{6)  yields  the  ML  frequency  objective  function. 

Jnif)  =  y'^KH.^y  -  y^Ki^^X  X'^K^^y  (B.8) 

This  function  must  be  minimized  with  respect  to  /  to  obtain  ML  estimates.  Once  f^i  is 
found,  then  but  is  found  via  Equation  B.7. 

B.2  Complex  Data  ML  Maximization 

Let  the  complex  number  6*  be  defined  by  real  scalers  bm  and  bn  so  that  =  biR+jbn 
and  let  ij  be  a  complex  number  defined  in  a  similar  faishion.  Let  J  be  a  function  of  a 
complex  variable,  6j.  The  goal  is  to  find  bi  so  that  dJ/dbi  =  0  indicating  a  stationary  point 
with  respect  to  bi  (40:517-519)  (24:890-894).  In  general,  J  is  not  an  analytic  function  and 
cannot  be  diflferentiated  with  respect  to  using  the  normal  rules  of  differential  calculus. 
Instead,  the  following  definition  will  be  applied  (40:517) 

21-1. 

dbi  ~  2 

Note,  the  derivative  is  zero  if  and  only  if  the  partial  derivative  of  each  of  the  real  and 
imaginary  parts  is  identically  zero.  Using  this  definition,  scalar  fimctions  defined  by  J 
can  be  minimized  over  bm  and  bn  simultaneously  (40:515-520).  Table  B.2  defines  the 
appropriate  scalar  derivatives  used  for  mziximizing  the  complex  ML  objective  function. 


dJ  .  dJ 
-J 


db, 


iR 


dbn 


(B.9) 
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Now,  let  6  =  6i  ...  6(5 


iT 


be  a  vector  of  complex  parameters  and  let  x  =  [xi  . . .  ig] 
be  a  vector  of  complex  constants.  Furthermore,  let  J{b)  be  any  scalar  function  of  b  and 
define  the  gradient  of  J  with  respect  to  6  as  the  vector 


dJ  dJ 
dbi  dbQ J 


T  T 


(B.IO) 


The  results  of  this  gradient,  applied  to  a  vector  product  and  a  vector  matrix  product,  are 
given  in  Table  B.2  and  these  definitions  will  be  used  to  minimize  the  ML  objective  function. 
With  9^  =  ^6^;  and  A  a  function  of  /  only,  the  likelihood  function  was  defined  as 


p{y-,e)  =  [27r]  ^\Kw\  ^exp[-(y  -  kb)^K^{y  -  A6)] 


(B.ll) 


Taking  the  natural  logarithm  yields  the  log-likelihood  function  L{y;  9)  as 


L{y-,9)  =  -Mln(27r)  -  ln|i(:,«|  -  [y  -  A6  K^^[y  -  Ab]  (B.12) 


H  . 


Assuming  Kw  is  known,  then  maximizing  L{y;  9)  is  the  same  as  minimizing 


Jc{e)  =  [y-^hY  k:[^ 


y-^ 


\b 


{B.13) 


Expanding  yields 


Jc{0)  =  y 


b^A^k^^il  -  y 


(B.14) 


Taking  the  gradient  with  respect  to  b  yields 


[VjJ(9)]  =  -  [a"K5,'»]'  +  [a''a:*'a6]' 


(B.15) 


Setting  equal  to  zero  and  solving  for  b  yields  the  ML  estimator  which  is  found  via  a  linear 
regression  on  y  provided  /  is  given. 
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Table  B.l  Derivatives  and  Gradients 


Real  Gradients 

Function 

Component  Vector 

Derivative  Gradient 

J  =  b^x  =  ^kbk 

J  =  b^Ab  =  E?=i  E?=i  bkakibi 

W,  ~  ~  ® 

M:  =  Efc=i  <^kibk  +  EILi  <^iibi  [Vft  J]  =Ab  +  A^b 

Complex  Scalar  Derivatives 

Function 

Component  Vector 

Derivative  Gradient 

J  =  bi  —  biR  +  jbii 

J  =  K=  biR-  jbii 

J  =  x*bi 

aj  _  1 
db,  2 

aj  _  1 

abi  2 

aj  _  If 
_26i _ IL 

a(biR+jbij)  •a(ftiR+jfti;) 

aftiR  ■'  dbii 

a(b,R-jbii)  ■a(biii-jbi!) 

[  aft.R  J  abi, 

XiR  -jXil]  -  ^[Xii  +jx, 

=  1  N/A 

=  0  N/A 

«]  =  i*  N/A 

Complex  Gradients 

Function 

Component  Vector 

Derivative  Gradient 

J  =  x‘*b  =  Yl'i^xKbk 

J  =  b''Ab  =  ZtiT,?=i 

SJ.  -  T* 
aft. 

U  =  ifa.,  =  [e?..  ilA 

V^(x"6)  =  x* 
Vf^{b^Ab)=\A^by 

Substitution  into  Jc{^)  yields  the  ML  frequency  objective  function. 


Jc{f)  =  y^K^^y-y^K;ilA 


w 


-i 


(B.16) 


(B.17) 


This  function  must  be  minimized  with  respect  to  /  to  obtain  ML  estimates.  Once  is 
found,  then  b^i  is  found  via  Equation  B.16. 
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Appendix  C.  Linear  Prediction  Coefficient  Constraints 

This  appendix  derives  the  Forward  and  Backwards  Linear  Prediction  (FBLP)  con¬ 
straints  and  the  symmetry  conditions  placed  on  the  LP  coefficients  for  the  real  and  complex 
forms  of  the  LP  model.  To  impose  the  sinusoidal  constraints  on  the  LP  coefficients,  define 
the  N  by  N  identity  matrix  as  In  and  the  N  hy  N  ‘Backwards’  identity  matrix  as  IBn, 
where  [IBn]  =  S[N  -  Jt  -  i  -I-  1].  Finally,  define  the  N  element  zero  vector  as  On  and 

IjJlc  =  2'Kfk- 


C.  1  Real  Sinusoids 

C.1.1  FBLP  Constraints.  Let  s[m]  be  the  sum  of  P  real  sinusoids  in  noise  and 
suppose  the  Forward  LP  coefficients  solve 

L 

^  a[/]s[m  — /]  =  0  (C.l) 

1=0 

for  L  >  P.  For  a  set  of  M  samples,  with  m  =  0 . . .  M  -  1,  substitution  yields 

a[0]cos(w*m-+-<^)  =  -  2]/=i  a[i]  cos(wtm -f  </>*)  cos(wife/) 

0  =  Y,i=i  sin(wjfem  -I-  0^)  s\n{u)kl) 

ioT  m  =  L ...  M  —  1.  Now  consider  the  Backward  LP  model  given  by  (87:418-422) 

L 

6[0]s[m  -  L]  =  —  ^2  —  L  +  1]  (C.3) 

(=1 

For  a  set  of  M  S2unples,  substitution  yields 


6[0]  cos(a;t[Tn  -  L]  +  4>) 
0 


-  Ef=i  cos(u;jt(Tn  -  L]  +  4>k)  cos(a;*0 

-  Ef=i  sin{wjfe[Tn  -  L]  +  (f>k)  sm{ujkl) 


(C.4) 
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for  m  —  L  =  0 . . .  M  —  L  -  1.  Thus,  with  b[l]  =  a[Z],  real  sinusoids  solve  both  the  forward 
and  backwards  LP  models. 


Z!i"=o ~  ^1  ~  0  tor  m  =  L  . . .  M  —  I 

d- /]  =  0  for  m  =  0. .. M-L  — 1 


(C.5) 


C.1.2  Symmetry  Constraints.  For  the  general  2P‘^  order  LP  model,  the  real  LP 
coefficients,  a[p],  solve 

2P  P  2P 

^  a[p\s[m  -  p]  =  0  =  ^  6fca[p]  cos{uJk[m  -p]  +  M  (C.6) 

p=0  *=1  p=0 


Expanding  the  real  sinusoid,  s[m],  yields 


P  2P 

53  h  cos(wfcm  +  53  <i[p]  cos(ukp) 

Li:=l  p=0 


+ 


P  2P 

53  h  sin(wim  +  </>*)  53  sin(a;jfcp) 

1*=!  p=0 


=  0 
(C.7) 


For  this  equality  to  hold  for  all  w*  and  m  requires  (8) 


2P 


2P 


53  ^b]  cos(a;jtp)  +  a[0]  =  53  ®  +  2a[0]  =  0 


p=i 


p=i 


2P  2P 

53  a[p]  sm{ujkp)  =  53  «b](^"‘'’  -  =  0 

p=i  p=i 


(C.8) 

(C.9) 


Solving  these  equations  simultaneously  yields 


Ep=i  obKe^^*”)  +  a[0]  =  0  and  a[p](e  +  a[0]  =  0 


(C.IO) 


Multiplying  the  second  summation  by  and  expanding  both  produces 


a[0]  +  a[l]e-''"‘  +  a[2]e’'"‘2  . . .  a[2P  -  +  a[2P]e-’'“‘2i’  =  o 

a[2F]  +  a[2P  -  +  a[2P  -  +  a[o]e^"‘2^  =  0 


(C.ll) 


Since  these  equalities  must  hold  for  all  Uk  then  a[p]  =  a[2P  -  p]  for  p  =  0 . . .  P.  Now 
let  a  =  [a[0]a[l] . . .  a[P]p  be  a  vector  containing  the  unique  LP  coefficients  and  let  o  = 
[a[0] . . .  a[2P]]^  be  a  vector  containing  all  the  LP  coefficients.  Then  a  is  related  to  a  via 
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the  linear  transformation  a  =  Ba  where 


(C.12) 


The  search  of  the  LP  coefficients,  contained  in  o,  of  a  order  model  has  been  reduced 
to  the  identification  of  the  P  unique  coefficients  contained  in  a. 


C.2  Complex  Sinusoids 

C.2.1  Forward  Backward  Linear  Prediction.  Let  s[m]  be  the  sum  of  P  complex 
sinusoids  and  suppose  the  Forward  LP  coefficients  solve 

L 

a[/]s(Tn  -  i]  =  0  (C.13) 

1=0 

for  L  >  P.  For  a  set  of  M  samples,  with  m  —  L . . .  M  -  I,  then 

L 

a[0ls[m]  =  -  ^  o[/]l[m  -  1]  (C.14) 

1=1 

Substituting  for  s[m]  gives,  for  each  frequency 

L 

a[0]e’“‘"’  =  -  (C.15) 

1=1 

for  m  =  L . . .  Af  —  1.  Now  consider  the  backward  LP  model  (87:418-422) 

L 

6[0]s[m  -LY  =  -Y,  W]slm  -  L  +  /]*  (C.16) 

1=1 

Substitution  for  the  complex  sinusoid  yields 

L 

1=1 
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(C.17) 


Letting  n  —  m  —  L  and  a[l]  —  6[/]  provides 


i 

a[/]s[n  +  /]'  =  0 

1=0 


(C.18) 


for  n  =  0 . . .  M  -  L  -  1.  Thus,  a  complex  sinusoid  solves  both  the  forward  and  backward 
LP  model 


J2iLo  ~  0  for  m  =  L  . . .  M  —  1 

YIiLq  +  /]]*=  0  for  m  =  0 . . .  M  —  1  -  L 


(C.19) 


C.2.2  Symmetry  Constraints.  Let  s[m\  be  the  sum  of  P  complex  sinusoids.  The 
set  of  P  complex  LP  coefficients,  a[p],  solve 


p  p  p 

a\p\3[m  -  p]  =  0  =  a[p]e~-’"‘'’  (C.20) 

p=0  /k=l  p=0 

To  be  zero  for  all  w*  and  6*  ^  0  requires 


Ep=i  +  a[0]  =  0  and  a[p]*e-'"‘P  +  o[0]*  =  0  (C.21) 

Multiplying  the  first  summation  by  and  expanding  yields 

a[0]eJ“‘^  +  . .  .a(F  -  l]e^"‘  +  a[F]  =  0 

(C.22) 

a[F]*e-'‘"‘^  +  a[F  -  . . .  a[l]*e-’‘^''  +  a[0]*  =  0 

Since  this  equation  must  hold  for  all  u;*,  this  implies  d[p]  =  d*[F  —  p]  for  p  =  0 . . .  F.  In 
addition,  for  F  >  1  complex  frequencies,  it  can  be  shown  (63)  when  u!k  =  7r(21  +  1) 
for  any  integer  I,  the  real  part  of  d[0]  =  0.  Conversely,  when  w*  =  27rl  for  any  integer 
I,  the  imaginary  part  of  d[0]  =  0.  To  impose  these  constraints,  let  F  be  an  even  number 
and  N  =  P/2,  then  a[N]  is  a  real  number  since  a[N]  =  Imposing  the  complex 

conjugate  symmetry  constraints  yields  the  real  vector  a  =  [a[0] . . .  a[F]]^  where 


a[0]  =  UflfiV]  (C.23) 

a[fc  +  1]  =  afl[A:]  for  A:  =  0  . . .  iV  -  1  (C.24) 

a[N  +  1  +  1]  =  a/[/]  for  /  =  0  . . .  iV  -  1  (C.25) 
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Only  F  +  1  coefficients  determine  a  completely.  The  LP  coefficient  vector  =  [a^,  aj] 
can  be  derived  from  a  as  a  =  B^a  where  Be  is  the  2F  +  1  by  P  matrix 


Off 
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1 

0]^ 

Ojf 

Oiv 

IBff 

On 

Off 

On 

In 

0 

Ojf 

0?; 

Off 

On 

-IB, 

(C.26) 


Finally,  if  P  is  an  odd  number,  let  AT  =  (F  -  l)/2.  The  symmetry  constraints  can  be 
imposed  by  defining  the  vector  a  as 


a[0]  =  aR[N] 

a[fc  4- 1]  =  afi[A:]  for  =  0 . . .  N  —  1  (C.27) 

a[N  + 1  +  l\  =  aj[l]  for  1  =  0...  iV 


The  LP  coefficient  vector  can  be  derived  from  a  as  a  =  Ba  where  B  is  the 

2F  +  1  by  F  matrix 


For  both  the  real 
of  the  LP  general 


B  = 


On 

In 

On 

On 

1 

Ojf 

0 

On 

1 

0l 

0 

On 

Oat 

IBff 

On 

On 

0^ 

On 

In 

On 

0 

Ojf 
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-IBff 

(C.28) 


and  complex  sinusoids,  imposing  these  constraints  yields  the  true  form 
linear  model 


Gia)YBa  =  G{a)Yca  =  e(a) 


(C.29) 
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Appendix  D.  Fixed  Point  Theory 

This  appendix  summarizes  the  Fixed  Point  theory  pertinent  to  this  dissertation. 

D.l  Definitions 

D.1.1  Linear  Space.  Let  jR  be  a  given  field  and  let  5  be  a  nonempty  set  with 
rules  of  addition  and  scalar  multiplication  which  assigns  to  any  x,y  G  5  a  sum  such  that 
X  +  y  €  5  and  to  any  x  6  5  and  c  €  jR,  a  product  such  that  cx  €  S.  Then  S  is  called  a 
linecir  space  over  R  if  the  following  properties  hold  (26:28),  (54:141): 

•  For  an  vectors  x,y,w  €  5,  then  {x  +  y)  +  z  =  x  +  (y)  +  z). 

•  There  is  a  vector  in  5,  called  the  zero  vector,  0,  for  which  x  +  0  =  x  for  any  vector 

in  S. 

•  For  each  vector  x  G  5,  there  is  a  vector  in  5,  denoted  -x,  for  which  x  +  (-x)  =  0. 

•  For  any  vectors,  x,  y  G  5,  x  +  y  =  y  +  x. 

•  For  any  scaleir  c  G  r  and  any  vectors  x,  y  in  S,  then  c(x  4-  y)  =  cx  +  cy). 

•  For  any  scalars,  Ci, C2  G  iZ  and  x  G  5,  then  (cj  +  C2)x  =  CiX  +  C2X. 

•  For  any  scalars,  Ci,C2  G  R  and  x  G  5,  then  (ciC2)x  =  Ci(c2x) 

•  For  the  identity  scalar  1  G  iZ,  then  lx  =  x  for  any  vector  x  G  5. 

D.l. 2  Euclidean  N-Space.  Euclidean  iV-space  consists  of  all  ordered  N  tuples, 
called  vectors  x  =  [ii . . .  x^v]^,  of  real  numbers  such  that  R^  =  {x|x„  G  iZ}  (58:18).  The 
i*'*  element  of  x  will  be  denoted  Xj.  Vector  addition  is  defined  as  z  =  x  +  y  so  that 
2j  =  Xj  +  Pi.  Scalar  multiplication  is  defined  for  a  G  iZ  as  z  =  ax  so  that  Zj  =  axj.  Using 
these  operations,  it  can  be  shown  Euclidean  N  space  is  a  vector  space  of  dimension  N  over 
the  field  of  real  numbers  (26:29). 
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D.1.3  Metric  Space.  Let  S  be  a  nonempty  set  and  let  x,  y  6  5.  A  distance 
function,  d{x,y)  G  R,  is  called  a  metric  on  S  if  the  following  hold  (33:1), (58:21) 


d{x,y) 

> 

0 

for  all  x,y  £  S 

d{x,y) 

= 

0 

iff  X  =  y 

d{x,y) 

— 

d{y,x) 

for  all  x,y  E  S 

d{x,  z) 

< 

d{x,y)  +  d{y,z) 

for  all  x,y,z  E  S 

For  Euclidean  N-Space,  the  distance  metric  is  defined  as 


d(x,  y)  =  11®  -  y||  =  .  -  Vi) 

\  «=i 


N 


(D.2) 


In  addition,  the  inner  product  is  defined  by  (®,  y)  =  lil/i  =  x^y. 


D.1.4  Complete  Metric  Space.  A  sequence,  {x*}  G  i?^,  is  called  Cauchy  if,  for 
every  real  number  e  >  0,  there  is  an  integer  N  such  that,  for  integers,  l,k  >  N  then 
II®*  ~  ®(||  <  £  (58:45).  A  metric  space  is  said  to  be  complete  if  and  only  if  every  Cauchy 
sequence  {x*}  C  converges  to  a  point  in  R^.  Since  a  sequence  {x*}  G  R^  converges  to 
a  point  in  R^  if  and  only  if  its  a  Cauchy  sequence,  Euclidean  A^-space,  under  the  distance 
metric  defined  above,  is  a  complete  metric  space  (58:45-46) 


D.1.5  Banach  Space.  A  normed  space,  5,  is  called  a  Banach  space  if  S  is  a 
complete  with  respect  to  the  metric  space  defined  by  d{x,y).  Thus,  the  Euclideam  space 
defined  above  is  a  Banach  Space. 

D.1.6  Open  and  Closed  Sets.  Let  .4  be  a  subset  of  R^ .  Then  A  is  an  open  set  if, 
for  each  x  G  A,  there  exists  some  real  number  c  such  that  for  each  y  G  R^  which  satisifies 
d{x,  y)  <  €,  then  y  €  A  (58:34-35).  A  set  B  C  R‘’^  is  said  to  be  closed  if  its  set  complement 
in  is  open  (58:37). 


D.1.7  Bounded  and  Compact  Sets.  Let  A  be  a  subset  of  R^ .  Then  A  is  bounded 
if  and  only  if  there  is  some  arbitrarily  large  constant  K  such  that  ||x||  <  if  for  every  x  G  A 
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(58:62).  Let  A  be  a  subset  of  the  Euclidean  space  .  Then  A  is  said  to  be  compact  if  A 
is  bounded  and  closed  (58:62). 

D.1.8  Continuous  and  Bounded  Functions.  Let  A  be  a  subset  of  .  Let  £  be 
a  mapping  such  that  £  :  A  — *  R^  and  let  Xo  €  Then  £  is  continuous  at  Xo  if  (58:79) 

lim  £(x)  =  £(x,)  (D.3) 

•C— ♦•Co 

Furthermore,  £  is  said  to  be  bounded  on  A  if,  for  every  x  G  A,  there  exists  some  finite  K 
such  that  ||£(x)||  <  K. 

D.1.9  Fixed  Point.  Let  Xg  G  S,  and  let  the  function  £  be  defined  on  S.  Then  Xg 
is  a  fixed  point  of  £  if  £(Xo)  =  Xg. 

D.2  Fixed  Point  Existence  Theorems 

D.2.1  Schauder’s  Fixed  Point  Theorem.  Let  5  be  a  compact  subset  of  a  Banach 
Space  and  let  the  fimction  £  be  a  mapping  such  that  £  :  5  — +  5  is  continuous  and  bounded 
on  5.  There  exists  a  fixed  point,  Xg  e  S  such  that  £(Xo)  =  Xg  (33:152). 

D.2. 2  Brouwer’s  Fixed  Point  Theorem.  Let  5  be  the  subset  of  R^  consisting  of 
all  vectors  x  such  that  ||xl|  <  1.  If  the  function  £  is  continuous  on  5,  then  there  exists  a 
fixed  point,  Xg  £  S  such  that  £(Xo)  =  Xg  (33:116). 
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Appendix  E.  Objective  Function  Gradients 


This  chapter  employs  the  identities  from  Appendix  A  to  derive  the  gradient  descent 
algorithms  for  minimizing  the  ML  frequency  objective  function  and  the  LP  coeflBcient 
objective  function. 


E.l  ML  Frequency  Objective  Function  Gradient 

From  Chapter  III,  the  ML  estimate  for  /  maximizes  an  objective  function  of  the 
following  form  function  J{f)  =  x^C~^x  where  f  —  [ji  ■■  ■  /p]^,  x  =  A^K^y,  and  C  = 
A^K^A.  Here,  only  A  is  a  function  of  f. 


Ai.  = 


f 


< 


cos[27r/j(Mo  -  i)| 
sin[27r/j(Mo  -  i)] 


0 


for  j  =  p 
for  j  =  P  +  p 
otherwise 


(E.l) 


for  p  =  1 . . .  P  and  i  =  \  . . .  M.  In  this  case,  the  partial  derivative  with  respect  to  /p 
becomes 

-27r(Mo  -  t)sin[27r/j(A/o  -  x)]  for  j  =  p 

— 27r(Mo  —  i)  cos[27r /j(Mo  -  i)]  for  j  =  P  +  p  (E-2) 

0  otherwise 

In  a  like  manner,  the  partial  derivative  with  respect  to  /p  and  /,  becomes 

— 47r^(Mo  —  iy  cos[2irfj{Ma  —  i)]  for  j  =  p  and  p  =  q 
in^{Ma  —  i)  sin[27r fj{Mo  —  x)]  for  j  =  P  +  p  and  p  =  q  (E.3) 
0  otherwise 

Thus,  using  the  results  from  Equation  A. 3,  and  Equation  A. 8 


r  a^A 
.3/pa/J,, 


'dA' 

.a/p. 


dx 

w. 


dC 

Wp 


dA' 

dfp. 


(E.4) 
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while,  from  Equation  A. 5  and  Equation  A. 9, 


&^x 

dUdf, 

di^C 

dUdh 


d^A 

dUdf, 

d^A 

dUdf, 

dAV 


KwV 


Kin  A  + 


flrl 

[oLl  [ 


"  [df, 
d^A 
dfM 


(E.5) 


(E.6) 


By  employing  Equation  A. 14  and  Equation  A. 15,  the  partial  derivatives  of  J{f)  with 
respect  to  fp  and  /,  can  now  be  written 


(E.7) 


(E.8) 


E.2  ML  Objective  Function  Gradient 

FVom  Chapter  3,  the  LP  objective  function  to  be  minimized  h^ls  the  form  J{a)  = 
x^C~^x  where  x  —  YcCX  and  C  =  A^K-wA.  Here  the  M  by  M  -  2P  matrix  A  is  defined 

£IS 

a[0]  0  ...  0 

:  a[0]  ...  0 

a[2P]  :  ...  0 

0  a[2P]  ...  a[0] 

0  0  0  a(2P] 

Now  define  the  matrix  Rp  as  the  M  by  M  —  2P  matrix  with 

1  for  i  =  j  +  p 

[Kp\i,j  =  1  for  t  =  2P  -  p  +  ji  (E.IO) 

0  otherwise 
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for  I  =  1 . . .  M  and  j  =  1 ...  M  —  2P.  Thus,  the  matrix  Rp  is  of  rank  M  for  p  =  0 ...  P 
and  so  is  the  sum  of  any  P  such  matrices.  With  a  =  [a[0]  . . .  the  M  ~  2P  square 

matrix  A  can  be  written  zis  a  rational  function  of  q  as 


A  =  Ro  +  Y,a\p\Rp 

P=1 


(E.ll) 


Finally,  define  Cp  as  the  vector  [ep)^  =  1  for  p  =  z,  and  zero  otherwise.  Using  these 
definitions  and  the  results  of  Equation  A.3  and  Equation  A. 5,  the  first  and  second  partial 
derivatives  of  x  with  respect  to  a  become 


dx  _  dYc  Y  _  Y 
da\p\  da\p]^  ^  ^\p] 

9a[p]5Q;[g)  5a [9]  ’’  'a [5] 


(E.12) 

(E.13) 


In  a  similar  fashion,  fi:om  Equation  A.8  and  Equation  A.9,  the  first  and  second  partial 
derivatives  of  C  with  respect  to  a  become 


da\p]da[q]  "  ^  [R^^wA  +  A^K^Rv]  =  R^KwR,  +  R^Kv,Rp  (E.15) 

Finally,  employing  the  results  of  Equation  A. 10  and  Equation  A. 11  for  the  inverse  and 
Equation  A.  14  and  Equation  A.  15  for  J{ot)  allows  the  first  and  second  partial  derivatives 
to  be  written  eis 


dJja) 

da[p] 

5a  [p]  9a  [9] 


■  r5C-> 

C  ^x  +  x^-r-rx 
da\p\ 


dx  1^5C-^ 
5a[p] .  5a[g] 

dx  1^5C-‘ 


5a[g]J  5a[p] 


X  +  2 

X  +  x 


dx 

.5a[p]  J 

.da{q]. 

T  d^C-^ 

- 2* 

da\p\a[q\ 

(E.16) 


(E.17) 
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Appendix  F.  Short  Time  Fourier  Transform/Filter  Bank  Equivalence 

This  appendix  shows  how  the  Short  Time  Fourier  Transform  (STFT),  evaluated  at 
K  equally  spaced  frequencies,  can  be  implemented  as  a  filter  bank  with  K  bandptiss  filters 
centered  at  a;*  =  2rklK. 

Let  t!;[n]  be  a  real,  finite  causal  window  of  length  N .  The  STFT  of  a  real  sequence, 
x[m],  over  this  window  is  defined  as  (65:714) 

N-l 

=  ^2  ui[n\x[m  +  (F-1) 

fi=0 

Usually,  the  STFT  is  evaluated  at  a  set  of  specific  frequencies,  a;  =  =  2'KklK,  for 

0  <  k  <  K  —  1.  As  such,  the  STFT  can  be  written  as 

N-l 

Xkim)  =  ^  to[n]x[m  +  nje"-’"'"  (F.2) 

fi=0 

Explicitly,  this  describes  the  STFT  obtained  by  sliding  x[m]  past  w[n]  one  sample  at  a  time 
so  that  the  calculation  of  Xk{m)  and  Xkim  +  1)  represents  an  overlap  of  -  1  samples 
of  x[m].  In  many  cases,  this  maximal  overlap  is  unnecessary  in  providing  an  accurate 
description  of  the  signal  in  the  time  window,  w[n].  Instead,  the  STFT  can  be  defined  in 
terms  of  the  overlap  between  two  successive  blocks  of  data  x[n]  as  (65:720) 

;v-i 

Xfc(Tn)  =  ^  ii;[n]x[i?Tn  +  (F.3) 

n=0 

Here,  there  is  an  overlap  of  N  —  fl  sample  points  in  the  calculation  of  Xkim)  and  A’*(rn  +  1). 

Now,  let  /io[n]  be  a  real,  causal,  low  pass  filter  of  length  N.  A  bank  of  bandpass  filters, 
centered  at  Uk  =  2'KklK,  can  be  constructed  by  modulating  the  prototype  coefficients  as 

=  (F.4) 


for  A:  =  0 ...  Ff  —  1.  In  this  case 


(F.5) 
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The  output  of  the  filter,  yjfc[m],  to  an  input,  x[m],  is  given  ais  the  convolution  sum 

ytM  =  f^k[n]x[m  -  n]  =  ^  ho[n\x[m  -  (F.6) 

n=0  n=:0 

Now  let  /loN  he  the  causal  version  of  the  reversed  window  i(;[n]  (65:716). 

/i^[n]  =  w[N  —  1  —  n]  (F.7) 

In  this  case,  the  output  of  the  A:"*  filter  can  be  written  as 

N-l 

y*[m]  =Y.w[N-1-  n\x[m  -  (F.8) 

n=0 

so  that 

N-l 

yt[m  +  AT  -  1]  =  ^  tefiV  -  1  -  n]x[m  +  N-\-  (F.9) 

n=0 

Letting  I  =  N  -  I  -  n  and  interchanging  the  order  of  summation  reveals 

yt[m  +  N-l]  =  '^  w[l]x[m  +  (p.io) 

/=o 

Factoring  the  extraneous  phaise  term  and  relating  to  Equation  F.3  shows 

y,[m  +  AT  -  1]  =  (F.ll) 

Now  let  the  output  of  the  k^''  channel  be  decimated  by  a  factor  R.  Then 

^•-l 

yk[Rm  +  N  -l]  =  J2  (F.12) 

1=0 

Factoring  the  extraneous  phase  term  and  relating  to  Equation  F.ll  shows 

y*[i?m  +  AT  -  1]  =  (F.13) 

These  equations  show  the  STFT,  with  a  data  overlap  of  N  -  R,  can  be  implemented  by 
decimating  by  R,  the  output  of  each  channel  of  a  filter  bank. 
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Appendix  G.  Complex  Random  Variables  and  Vectors 


This  appendix  provides  the  definitions  of  complex  random  variables  and  vectors  and 
discusses  their  general  properties. 


G.l  Random  Variables 

Let  and  represent  real  samples  from  separate  random  time  series  with  the 
joint  PDF,  p(u/j[A:],  u/[fc]).  Let  ii[A:]  be  the  complex  representation,  u[k]  — 

The  squared  two-norm  of  u[k\,  denoted  |u[fc]p,  is  defined  as  (40:500-501) 

|u[A:]p  =  u[A:]*u[A:]  =  -I-  (G-1) 

where  the  *  denotes  complex  conjugation.  The  distribution  function  for  u[fc]  is  defined  in 
terms  of  the  joint  distribution  of  UR[k]  and  u/[fc].  That  is,  for  a  fixed  value  u°  =  Ur  +juf, 
the  distribution  function  is  defined  as  (87)[l7-47] 

F(i(u°)  =  Pr(u[l:]  <  u°)  =  P{uR\k]  <  u°R]Ui[k]  <  u°j)  (G.2) 


and  is  related  to  the  density  function  /u(u)  zis 

/-“R  /■“? 

=  /  U{u)du=  /  f^^^^,{uR,ui)duRdui 

>/  — 00  J  —  oo  •/  — 00 


(G.3) 


Now  let  ’i'  be  the  region  defined  by 


^  :=  {u°  <  u[fc]  <  =  {ufl  <  Ur[A;]  <  ■UrIPIIu/  <  U/IA:]  <  u)}  (G.4) 

The  probability  that  u[fc]  €  is  defined  as 

P{u[k]e^)=[  fii{u)du=  f  f  SuH,n,{uR,Ui)duRduj  (G.5) 

y ti® 

The  mean  of  u[fc]  is  defined  cis  the  expected  value  of  each  component 


m[k]  =  E{u[k\}  =  -f- j£;{u/[A:]}  =  mR[A:]  +  jmi[k] 


(G.6) 
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The  variance  of  u[fc]  is  defined  as  (40:500-501) 


V{u[k]}  =  =  E{\u[k]\^}  -  |m[fc]p  =  VKIA:]}  +  V{u,[A:]}  (G.7) 

The  correlation  between  u[k]  and  u[l\  is  defined  as 

(G.8) 

and  describes  how  well  the  values  of  tt[Ti]  at  time  samples  k  and  I  track  each  other  (87:141). 
In  particuleir,  if  rii,u[A:,  1]  is  zero,  then  u[A:]  and  u[f]  are  said  to  be  uncorrelated.  Additionally, 
since  F{ii[jfc]}  is  equal  to  the  sum  of  the  variance  of  each  component,  then  Uit[k]  and  Ui[k] 
are  uncorrelated  at  the  fc''*  sample.  The  covariance  of  u[k]  and  u[/]  is  defined  as 

Cov{tl[A:],it[l]}  =  E{{u[k]  -  m[A:])(ti[f]  -  m[/])*}  =  £'{u[fc]ii[/]*}  -  Th[fc]’m[/]  (G.9) 

In  general,  if  u[k]  and  u[/]  are  independent,  then  Cov{u[fc],  u[/]}  =  0.  The  converse  is  not 
necessarily  true  (86:56). 

G.2  Random  Vectors 

Let  Uu  =  [ur[M  —  1] . . .  Ufl[0]]^  and  «/  =  [u/[Af  —  1] . . .  u/IO]]^  be  random  vectors 
obtained  from  two  different  time  sequences.  The  complex  vector  representation  for  the 
two  sequences  is  defined  as  ti  =  ur  +  jui  and  the  squared  two- norm  of  u,  denoted  |up  is 
defined  as  (87:22-39) 

\u\l  =  u^u  =  uJiUr  +  uJui  (G.IO) 

where  the  H  operation  indicates  complex  conjugate  transpose.  The  distribution  function 
for  u  is  defined  in  terms  of  the  joint  distribution  of  Ur  and  Uj.  For  a  fixed  vector  u°  = 
Ur  -h  ju°i,  the  distribution  function  is  defined  as 

Fu{u°)  =  Pt{u)  <u°  =  P{ur  <  u°R,  U[  <  u°i)  (G.ll) 
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and  is  related  to  the  density  function 


fU°  /•««  fW} 

Fui'^°)=  fui^)du=  /  fu^.u,{uR,Ui)duRdui  (G.12) 

7  —  00  7  —  00  7  —  00 


Let  be  the  region  defined  by 


^  :=  {u°  <u<u^}  =  {Ufl  <  tifl  <  ujj  Plti®  <  U/  <  u}} 


(G.13) 


The  probability  that  u  €  is  defined  as 


P(ti6^')=/  fu{u)du=  [  f  Sun,u,{^R^'^i)duRdui  (G.14) 
Ju°  Ju%  Ju° 

The  expected  value,  or  mean  m,  of  a  complex  random  vector  is  defined  as  the  expected 
value  of  each  component 


m  =  E{u}  =  Pfufl}  +jE{u!} 


(G.15) 


where 


fca 

•^{[*1"}  =  /  (ufiN  +j^{H)fuK,u,{uR,Ui)duRdui  =  mR[n]  +  jmi[n]  (G.16) 
7  —  00 


The  correlation  matrix  for  u  is  defined  ais 


^  —  E{uu  }  (G.17) 

so  that  =  7‘u,u[A:, /].  If  u[k]  and  t2[/]  are  uncorrelated,  then  Ru  y,  is  a  diagonal 

matrix  with  the  variance  of  each  time  sample  on  the  diagonal.  The  covariance  matrix  of 
u,  denoted  K^,  is  defined  as 


Kfi  =  E{{u  -  Th){u  -  m)“}  (G.18) 

so  that 

-  m[/])*}  (G.19) 
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Finally,  let  A  be  an  by  M  complex  matrix  of  full  column  rank  and  let  6  be  an  by  1 
complex  vector.  Then,  if  y  =  Ati  +  b,  where  u  is  a  random  vector  with  a  mean  of  and 
a  covariance  matrix,  then  y  has  a  mean,  denoted  thy,  and  a  covariance  of  matrix, 
denoted  Ky,  of  (40:502) 


rhy  =  Am 


u 


+  b  and 


ky  =  AKuA^ 


(G.20) 


G.3  Wide  Sense  Stationary 

A  random  process  is  said  to  be  Wide  Sense  Stationary  (WSS),  if  the  expected  value 
of  u[n]  is  independent  of  n  and  the  correlation  function  function  is  a  function  only  of  the 
spacing  between  samples  (87:140-155). 

E{u[n\}  =  fi  a  constant  and  ruii[A:, /]  =  ruu[fc  -  i]  (G.21) 

Now  let  ti  be  a  random  vector  with  correlation  matrix  Rfi  u-  If  «  is  a  WSS  sequence,  then 

[Ru.uki  =  -l]=  r^u[l  -  fc]*  (G.22) 

This  correlation  matrix  is  an  Hermitian  Symmetric,  Toeplitz  matrix. 
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Appendix  H.  Complex  Normally  Distributed  Random  Vectors 


This  appendix  describes  how  to  represent  a  real,  normally  distributed  random  vector 
as  a  complex  normally  distributed  random  vector.  Let  the  vector  u  be  the  concatenation 
of  the  M  dimensional  real  vectors  Ur  and  tt/  so  that  =  \u\,u^].  Furthermore,  let  u 
be  normally  distributed  with  mean,  mu-,  and  covariance,  Ku,  defined  by 


muR 

;  Ku  = 

A 

-B 

mu  — 

mu, 

B 

A 

(H.l) 


where  A  =  KurUr  =  Nu,u,  is  a  symmetric  matrix  and  B  =  Ku, Ur  =  -KurU,  is  a  skew 
symmetric  matrix.  The  PDF  for  u  can  be  written  as 


p{u)  =  [2-k]  ’^IKul  '''^exp{-.5a:^A'u^x} 


(H.2) 


where  x  =  tt  —  niu-  From  this  form,  the  inverse  of  Ku  can  be  written  as  (40:556-572) 


K 


u 


E  -F 
F  E 


(H.3) 


where  E  =  {A  BA  ^B)  ^  =  E^  and  F  =  —{A  -I-  BA  ^B)~^BA  ^  =  —F^.  Then,  since 
KuKu  =  I  =  KuKu,  the  following  identities  apply: 


AE  -  BF  =  I  =  EA-  FB  and  BE  +  AF  =  0  =  FA  + EB  (H.4) 

Using  these  identities,  the  aurgument  of  the  exponential  becomes 

x^KuX  =  x\Exr  -I-  xJFxr  -  x\Fxi  -I-  xjExi  (H.5) 

Now  define  u  =  Ur+  juj  and  let  the  complex  covariance  matrix  of  u  be  =  A  +  Bj. 
Furthermore,  assume  the  form  of  the  inverse  can  be  written  as  [Qu]  =  C  -f  Dj.  Since 
QiiQl'  =1  =  I  =  AC -BD  +  j{AD  +  BC)  =  {CA  -  DB)  +  j{DA  CB). 
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From  Equation  H.4,  C  =  E  and  D  =  F  so  that  =  E  +  Fj.  Thus 

x^Q^x  =  x\ExR->rx]Exi  -  x\Fxi  +  x]Fxr 

+  j{x]iFxR  -  xJExr  +  x'^Exi  +  xj Fxi)  (H.6) 

Since  F  =  -F^  and  E  =  E^  then  x\Fxr  =  xjFxj  =  0  and  -xJExr  +  x\Exi  =  0. 


Hence,  from  Equation  H.5 

x^Q^x  =  x^K:{^x  =  x\Exr  +  xjExj  -  XrFxj  +  xJFxr  (H.7) 

Finally,  the  determinant  of  the  covariance  matrix  can  be  written  as  (40:572) 

\Ku\  =  \A\\A  +  BA-^B\  (H.8) 

But,  A  +  BA~^B  —  (A  -  j B){A~^){A  +  jB)  so  that 

liful  =  =  |<5*P  (H.9) 

Thus,  the  representation  of  the  complex  form  of  u  is  given  eis 

p{u)  =  [2‘n]~‘^\Qu\~''  exp{-.5(Ti  -  rhu)^Q'^{u  -  m*)}  (H.IO) 

where  =  Kur,Ur  +  jKu,,UR-  —  ^Qii  Then  (83:214) 

\Qu\  =  ‘i-^\Ku\  (H.ll) 

The  complex  form  of  u,  that  is  «,  is  given  ais 

p{u)  =  [7r]"^|Ku|-‘  exp{-(ti  -  mu)^k:^{u  -  m^)}  (H.12) 
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Here  =  2{Kuk,Ur  +  jJ^Ui.Ur)-  each  element  of  the  covariance  matrix,  for  k,l  = 
1 ...  M,  then  U/  and  ur  are  uncorrelated  since 


Cov{ufl[fc]u/j[l]}  =  Cov{u/[fc]u/[l]}  and  Cov{u/[fc]u/j[/]}  =  -Cov{ufi[fc]ii/[/]}  (H.13) 
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Appendix  I.  Alternate  Architecture 


This  appendix,  shows  how  the  proposed  method  of  implementing  the  nonmeiximally 
decimated  polyphase  UDFT  filter  bank,  though  architecturally  different,  is  mathematically 
related  to  the  method  proposed  by  Rabiner  and  Crochiere  when  the  decimation  factor  M 
is  related  to  the  number  of  filters  K  by  K  =  FM  for  integer  F  (55). 


I.l  New  Architecture 

As  shown  in  Figure  4.3  the  new  architecture  for  implementing  the  nonmaximally 
decimated  UDFT  filter  bank  is  to  zero  pad  eaoh  polyphase  component  filter  with  F  —  I 
zeros  and  decimate,  by  M,  the  input  signal  across  all  K  channels  prior  to  taking  the  IDFT 
of  the  filter  outputs  at  time  n.  Let  /lofnj  be  a  causal  low  pass  filter  of  length  N.  This  filter 
can  be  decomposed  into  K  polyphase  components  Ek{z)  as  (92) 

K-l  oo 

^  z~‘‘Ek{z^)  where  Eit(z^)  =  Y^ho[nK  +  (I.l) 

k=0  n=0 

In  terms  of  the  time  domain,  the  filter  coefficients  can  be  written  as 


e*H 


1ha[Kn/F  +  A:]  for  n  =  pF  for  integer  p 
0  otherwise 


(1.2) 


Since  e*  [n]  is  nonzero  only  for  integer  multiples  of  F  the  output  of  each  polyphase  compo¬ 
nent  can  be  written,  without  any  loss  of  generality,  as 

OO 

^*H  =  21  k]x[Mn  -  k  -  Ki]  (1.3) 

»=o 

At  time  n,  the  output  of  the  /*'*  channel  is  the  IDFT  of  the  sequence  tjt[n]  so  that 

A”— 1  oo 

r[/]  =  y,[n]  =  [Ki  +  k]x\Mn  -k-  Ki]e^^^'‘‘l^  (1.4) 

fc=0  i=0 

This  is  equivalent  to  calculating  the  Short  Time  Fourier  Transform  (STFT)  of  x[n]  over  the 
window  i(;[n]  =  /io[-n]  ,  overlapped  with  N  —  M  data  points,  and  evaluating  at  w  =  2rl/K. 
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1.2  Previous  Architecture 


As  shown  in  Figure  I. la,  in  the  method  proposed  by  Rabiner  and  Crochiere,  the 
data  is  decimated  by  K,  expanded  by  F,  and  filtered  with  K  polyphase  filters  prior  to 
performing  the  IDFT  on  these  filter  outputs  (55).  Alternatively,  as  shown  in  Figure  I. lb, 
this  implementation  is  equivalent  to  partitioning  the  polyphase  filter  coefficients  into  F 
separate  blocks,  indexed  by  1  <  /  <  F.  Each  block  is  comprised  of  M  separate  branches, 
indexed  by  0  <  m  <  M  -  1,  with  the  branch  polyphase  filter  Pm{^)  given  by 

Pm[n]  =  h[Mn  +  m]  for  n  >  0  (1.5) 

The  component  of  the  polyphase  representation  for  0  <  fc  <  AT  —  1  can  be  evaluated  in 
terms  of  the  block  index  I  and  the  branch  index  m  as  k  —  M{1  —  1)  +  m.  The  data  entering 
the  m''*  branch  of  the  1**  block,  i(  mH,  is  simply  the  signal  i[n]  shifted  by  M(/  —  1)  +  m. 

~  Af(^  -  1)  -  m]  (1.6) 

This  data  is  then  decimated  to  produce  the  sequence 


Ui,mH  =  xi^„[Kn]  =  x[Kn  -  M{1  -  1)  -  m] 


(1.7) 


Upon  expanding  by  the  factor  F,  the  new  sequence  becomes 


U(_„,[n/F]  when  n  =  Frii  for  integer  ni 
0  otherwise 


Thus,  the  output  of  the  m^'*  branch  of  the  I*'*  block  can  be  written  as 


00 

i=0 

Taking  into  account  the  F  —  1  zeros  between  the  samples  of  gives 


(1.8) 


(1.9) 


where  /;  =  0, 1, . . . ,  F  -  1  and  n  =  Fnj  for  integer  ui.  In  terms  of  the  prototype  filter, 


S(,m[Fn(  +  /,]  =  ^  ho[Ki  +  Mfi  +  m]x[Kni  -  Ki  -  M{1  -  1)  -  m]  (I-ll) 

i=0 

Since  gi.mN  is  just  S(,m[u  -  F  +  1],  this  output  can  be  written  as 

OO 

qi,m[Fni  +  //  +  F  -  /]  =  ^  ho[Ki  +  Mft  +  m]x{Kni  -  M{1  -  \)  -  Ki  -  m]  (1.12) 

i=0 

At  a  specific  time,  n,,,  the  IDFT  is  performed  over  the  outputs  from  the  polypheise  filters. 
Then,  for  time  synchronization  of  each  delayed  filter  output, 

Fni  +  fi  +  F  -  I  =  Ho  (I'13) 

for  I  =  1, 2, ...  F  and  //  =  0, 1, . . .  F  —  1.  From  the  first  block,  /  =  1,  of  M  filter  outputs 

Fni  +  /i  -  1  =  Fni  +  /,  -  I  (1.14) 


Since  0  <  /(  <  F  -  1,  then,  for  time  synchronization,  nj  =  ni  +  giF,  where 


9t  =  < 


0  for  /  <  F  -  /i 
1  for  1  >  F  +  1  —  /i 


(1.15) 


so  that  fi  =  fi  —  l+l  —  9iF.  The  output  of  the  IDFT  channel  at  time  Uq  is 

F  M-l 

Qp[no]  =  E  E 


(1.16) 


(=1  m=0 


Conversely,  in  terms  of  k,  the  output  of  the  channel  of  the  IDFT  can  be  written  as 

QpK]  =  (1-17) 


*=0 
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1.3  Architecture  Relationship 

This  section  derives  the  relationships  between  the  two  architectures.  From  Equa¬ 
tion  1.3,  for  tjt[n],  let  k  =  Mfi  -I-  m  and  n  =  Fni  fi  -  I  Substitution  yields 

00 

tMS,+m\Fni  +  fi-l  +  l\  =  'Y^ho[Ki  +  MfiA-m]x[M{Fni-l  +  l-¥fi)-Ki-Mfi-m\  (1.18) 

i=0 

Thus,  referring  to  Equation  1.12,  shows 


tMf,+m[Fni  +  /(  —  ^  +  1]  —  qi,m[F ni  +  F  -  I  +  fi]  (1-19) 


Making  the  substitution  for  rio  =  Fni  +  F  —  I  +  fi  and  using  Equation  1.15  with  I  =  1 
yields 

9/,m[^o]  —  tM{fi—gtF)+m{Frii  "b  /i]  (1.20) 


Using  k  =  M{1  -  1)  -I-  m  gives 


9*  ho]  =  { 


tMfi+k[Fni  +  /i] 
tk-M{F-fi){Fni  -f-  fi] 


for  0  <  fc  <  M{F  -  /i)  -  1 
for  M{F  -fi)<k<K-l 


(1.21) 


or  equivalently. 


9*  ho]  =  tmod{Mfi+k)K[F''T'i  +  /i] 


(1.22) 


This  shows  the  outputs  of  the  chaimels  of  the  K  polyphase  filters  proposed  be  Rabiner  and 
Crochiere  are  time  related  to  the  outputs  of  the  new  method.  At  tIo,  with  Qpho]  =  Q[p] 
and  T[p]  =  yp[Fni  /J,  the  IDFT  output  is 

Q[p]  E  and  T[p]  =  ^  (1.23) 

*=o  fc=o 

Using  the  MODk  operation  defined  by  Equation  1.22  gives 


Q[p]  = 


(1.24) 
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The  outputs  of  the  K  filter  banks  introduced  by  Rabiner  and  Crochiere  are  related,  via  a 
time  dependent  phase  term,  to  the  outputs  of  the  nonmajcimally  decimated  UDFT  filter 
bank. 
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a:  Current  Architecture 


.-1-  i  r 


BLOCK  1 


V : 

— -  c^mi 

Q  [n] 

1-1  ; 

q,  [n]  : 

l/n  j 

Q  [n] 

M 

:  » 


»  BLOCK  2 


_  V  [n]  '  s  [r 

4  l.m  ;  I  l,m 

t  F  - !  P  (z)  I - 

!  M-1 


q,[n]:  IDFT!  QJ!? 


- ^  u  [n]  -  V  [nj  - 

j  I  I  - 

^  P^j(2) 


s  [n]  - :  q,  [n]  !  j 

;  _  -(F-l)  ' 


Q  [n] 

,  (F-1)M 


BLOCK F 


b:  Equivalent  Architecture 

Figure  I.l  Conventional  Nonmaocimally  Decimated  UDFT  Filter  Bank 
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Appendix  J.  Phase  Noise  Analysis 


This  appendix  shows  the  covariance  matrix  of  the  noise  introduced  to  the  phase  term 
of  a  filtered  complex  sinusoid  will,  in  general,  depend  on  the  frequency  of  the  signal  to  be 
estimated. 

Let  th  be  a  zero  mean  normally  distributed  complex  noise  vector  with  covariance 
matrix,  K^ju,  derived  from  M  samples  of  two  noise  sequences,  Wi  =  [wi\M  — 
for  i  =  R,  I  so  that  w  =  Wr  +  jwi.  A  single  complex  sinusoid  embedded  in  this  noise  be 
written  as 

y[m]  =  +  w[m]  =  ^  (jj) 

Here  v[m]  =  VR[m]  +  jvi[m]  =  [^6i "* j  w[m].  Assuming  the  magnitude,  &i,  is  large 
in  relation  to  the  magnitude  of  the  noise  samples  for  all  m,  the  magnitude  and  phase  of 
[1  4-  i3[m]]  can  be  written  as 

|1  4-i5[m]|  =  ■y/(l  +Ufi[mj)2  +  v/[Tn]2  «  1 
Z{1  +  uH}  =  -  tan-'{Yf^}  «  v/[m] 

Now,  expanding  vi[m]  yields 

vAm]  =  ^  [cos(27r/im  +  4>i)wi[m]  —  sin(27r/im  +  <;6i)ryfl[Tn]]  (J.3) 

Ol 

For  M  samples,  Vi  =  [v[[M  —  1] . . .  u/[0]]^  can  be  written  as 

Vi  =  ^[Cwi  -  Swr]  (J.4) 

Ol 

where  C  and  5  are  diagonal  matrices  with 

Cii  =  cos{2nfi[M  —  i]  +  i^i)  (J-^) 

Sii  =  sin(27r/i[M  —  t]  +  0i)  (J.6) 

Since  Wi  and  Wr  are  zero  mean,  normally  distributed  random  vectors  with  covariance  ma¬ 
trices,  KwjWn  Nwr,Whi  and  Kuo^  Wr,  the  vector  Vi  is  a  zero  mean,  normally  distributed 
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random  vector  with  covariance  matrix 


Kv,  =  7-  [CKy},w,C^-CKxi),.W!tS^-SKwR,w,C^  +  SKw„,WRS^] 

.0l 


In  general,  the  covariance  matrix  of  Vj  is  dependent  on  /i,  the  frequency  to  be  estimated. 
For  the  special  case  when  wrItti]  and  wi[m]  are  uncorrelated  and  independent  random 
variables,  then 

£7^ 

Kw,,w,  =  Kwft.WR  = 

Kwr,W,  =  Kw,,Wri  —  [0]  (J-9) 


The  covariance  matrix,  K-u,,  becomes 

and  is  independent  of  /i . 
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